Method and device for identification and/or sorting of medicines

ABSTRACT

A medicine identification and sorting system is disclosed, which includes an image capturing system for creating a digital image of at least a portion of a target medicine, and an image processing system for comparing said created target medicine image with reference medicine images in a reference medicine image database to identify and/or then sort the medicine from a mixture of medicines.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. application Ser. No.14/555,605, filed Nov. 27, 2014, now U.S. Pat. No. 9,940,439, whichclaims the benefit of U.S. Provisional Application No. 61/910,283 filedNov. 29, 2013, the entire disclosure of each of which is herebyincorporated herein by reference.

FIELD

The subject technology relates to a system as well as to a method anddevice for identifying and/or sorting mixed medicines of different typesfor proper disposal.

BACKGROUND

Every year, a large percentage of medicines (including prescription andover-the-counter) sold remain unused in households or hospitals, arerecalled or become expired. For example, in one county (King county, WA)in the U.S., the amount of unused household medicines adds up to about11 million containers per year. Unwanted medicines that aren't properlydisposed of end up in the environment. Currently, the two main modes ofdisposing of unused medicines are by flushing them down the toilet orthrowing them in trash. Both of these methods contribute to thecontamination of soil and the underground or surface water supplies.Wastewater treatment processes do not remove pharmaceuticals containedin medicines. Most pharmaceuticals are not biodegradable and thusexposure to air or sun in landfills does not deactivate or degrade them.As the results, these chemicals accumulate in the environment and inaggregate can be hazardous to the environment and human health. Forinstance, an Associate Press investigation has found that in 24 majormetropolitan cities(hosted.ap.org/specials/interactives/pharmawater_site/) in the U.S. thedrinking water supplies of at least 46 million people are contaminatedwith minute amounts of many pharmaceuticals. Some of frequently detectedpharmaceuticals in this investigation were atenolol (heart medication),carbamazepine (for seizure), gemfibrozil (anti-cholesterol), meprobamate(tranquilizer), naproxen (over-the-counter pain reliever), phenytoin(anti-seizure medication), sulfamethoxazole and trimethoprinm(antibiotics).

At the moment, there is no way of knowing what impact low levels ofpharmaceuticals will have on human health. However, there is no doubtthat a cumulative effect of so many different drugs (includinganti-psychotic drugs, antibiotics, pain medications, heart andcirculation drugs, contraceptive drugs, diabetes drugs, andchemotherapeutic or anti-proliferative drugs) can have adverse effect onhuman health. Therefore, there remains a need for a safe and properdisposal of medicines.

SUMMARY

The first step in proper disposal of medicines is a proper separation.The subject technology provides a method and a device for effectiveidentification and separation of medicines when these medicines are notin their original containers and are loose and mixed with other types ofmedicines.

The balk of almost all medicines is made up of excipients, or inactivesubstances that are formulated alongside the active pharmaceuticalingredient (API or AI). In general, excipients are substantially lessharmful than APIs. The most environmentally effective means for properdisposal of medicines therefore includes first separating excipientsfrom APIs and then properly disposing of the APIs by incineration,recycling, chemical modification or the like. Thus, the first step inproper disposal of medicines is their proper identification and/orseparation. Separated medicines can then be subject to furtherprocessing for isolating their active ingredients.

The majority of the unused medicines and medications that eventuallyfind their ways into the environment are in form of medicines or oraldosage forms (e.g., syrups). Oral dosage forms are usually the mostconvenient choice for taking medicines. As known worldwide, taking amedicine via oral route is one of the best options as it's the simplestand easiest way for any patient to take a medication. Other frequentlyused forms of medicines are injectables (e.g., ampules), suppositoriesand inhalers. As such, there exists a need for a system and method forrapid identification and/or sorting of dosage forms of differentphysical attributes and characteristics such as size, shape, color, etc.

Thus, in one aspect, the subject technology relates to a medicineidentification system that can identify and/or sort a target medicinefrom a mixture of medicines; including: (a) an image capturing systemfor creating a digital image of an least a portion of a target medicineand (b) an image processing system for comparing said target medicineimage with reference medicine images in a reference medicine imagedatabase. In an embodiment relating to this aspect, the image capturesystem and said image processing system further includes a handheldelectronic device.

In another aspect, the subject technology relates to a medicineidentification system that can identify and/or sort a target medicinefrom a mixture of medicines; including: (a) an image capturing systemfor creating a digital image of an least a portion of a target medicine;(b) an image processing system for comparing said target medicine imagewith reference medicine images in a reference medicine image database;and (c) a sorting system having a medicine identification chamber andsorting elements for routing said target medicine from saididentification chamber to a desired location after said target medicineimage is processed or identified. In an embodiment relating to thisaspect, the image capturing system, said image processing system andsaid sorting system include a handheld or portable device.

In another aspect, the subject technology relates to a medicineidentification system that can identify and/or sort a target medicinefrom a mixture of medicines; including: (a) a first sorting system forsorting medicines based on their width or diameter as they move radiallywithin a sorter of the first sorting system; (b) an image capturingsystem for creating a digital image of an least a portion of a targetmedicines; (c) an image processing system for comparing said targetmedicine image with reference medicine images in a reference medicineimage database; and (d) a second sorting system having a medicineidentification chamber and sorting elements for routing said targetmedicine from said identification chamber to a desired location aftersaid target medicine image is processed or identified.

In another aspect, the subject technology relates to a method foridentifying and sorting a target medicine from a mixture of medicinesincluding an image capturing step for creating at least one digitalimage of the target medicine, an image processing step for comparing theat least one digital image with reference images and identifying thetarget medicine by determining a match between the at least one digitalimage and a reference image, and a sorting step for routing said targetmedicine to a desired location after the target medicine is processed oridentified.

In an embodiment relating to any of the above aspects, the methodfurther includes a first queuing and/or sorting step for queuing orsorting of the mixture of medicines before or during their introductioninto the image-capturing step. Alternatively or in addition, the firstqueuing and/or sorting step is performed by a first sorting apparatus,the image capturing step is performed by an image capturing apparatus,the image processing step is performed by an image processing apparatus,and the sorting step is performed by a second sorting apparatus.Alternatively or in addition, said first queuing and/or sorting steparrange the medicines such that they are introduced one-by-one to theimage capture step. Alternatively or in addition, the first queuingand/or sorting apparatus include a plurality of ramps designed to engagewith and route the medicines towards the image capturing apparatus.

Alternatively or in addition, the image capturing apparatus includes adigital camera. Alternatively or in addition, the image processingapparatus includes a central processing unit, a main memory, and astorage unit; wherein the storage unit further comprises a database ofthe reference images comprising digital images of at least a portion ofreference medicines. Alternatively or in addition, the second sortingapparatus includes flapper elements controlled by electric motors.Alternatively or in addition, the determining of a match between the atleast one digital image and the reference images is based on a matchprobability output or percent identity between the at least one digitalimage and the reference images.

Alternatively or in addition, the reference images includes images ofknown medicines produced under a similar condition as the targetmedicine. Alternatively or in addition, the comparing of the at leastone digital image with the reference images include comparing of atleast a physical feature or portion thereof between these images;wherein the physical feature comprises shape, color, surface line,imprint, marking, deboss, emboss, groove or writing. Alternatively or inaddition, the image processing step is carried out by detecting edgesand lines in the at least one digital image and comparing said edges andlines with those in the reference image and determining if a match isfound between said target medicine and the reference image foridentifying the target medicine.

Alternatively or in addition, said image processing step furtherincludes: adjusting said target medicine image based on skew or angle ofsaid target medicine; scaling said target medicine image to matchapproximate size of said reference image; blurring said target medicineimage; finding said target medicine image edges with an edge detectoralgorithm; finding said target medicine image lines with a linetransform algorithm; marking said lines and edges into a modified targetmedicine image; overlaying said modified target medicine image over thereference image.

Alternatively or in addition, the method further includes the steps of:classifying said target medicine in an output category based on asuccessful or unsuccessful match with said reference image.Alternatively or in addition, the method further includes the steps of:utilizing a remote server and network connection to store said referenceimages. Alternatively or in addition, the method further includes thesteps of: utilizing a remote server and network connection to store saidreference medicine images and to carry out said image processing step.

Alternatively or in addition, determining if a match is found betweensaid target medicine and said reference image further includes the stepsof: using several matching algorithms in an iterative fashion todetermine a match with highest probability or nonmatch with highestprobability. Alternatively or in addition, said edges and lines with areference image further includes: normalizing said match determinationoutput; localizing regions of higher matching probability foridentifying marks; recognize identifying marks using optical characterrecognition.

In another aspect, the subject technology relates to a medicineidentifying device for identifying and sorting a target medicine from amixture of medicines, including: an identification chamber by or withinwhich a target medicine image is captured and thereafter processed foridentification, wherein said identification chamber comprises an imagecapturing apparatus for creating at least one digital image of thetarget medicine; an image processing apparatus for comparing the atleast one digital image with reference images and determining a matchbetween the at least one digital image and a reference image foridentifying the target medicine.

In a related embodiment, the device further includes a first queuingand/or sorting apparatus for queuing or sorting of the mixture ofmedicines before or during their introduction into the image capturingapparatus, and a second sorting apparatus for routing said targetmedicine to a desired location after the target medicine is processed oridentified. Alternatively or in addition, said image processingapparatus includes a central processing unit, a main memory, and astorage unit; wherein the storage unit further comprises a database ofthe reference images comprising digital images of at least a portion ofreference medicines.

Alternatively or in addition, the determining of a match between the atleast one digital image and the reference images is based on a matchprobability output or percent identity between the at least one digitalimage and the reference images. Alternatively or in addition, thecomparing of the at least one digital image with reference imagesincludes comparing of at least one physical marker includes shape,color, surface line, imprint, marking, deboss, emboss, groove or writingor a portion thereof in the images. Alternatively or in addition, thedevice further includes: a network; a network interface unit; and aremote server for storing a database of reference images. Alternativelyor in addition, the device is handheld. Alternatively or in addition,the device further includes a queuing apparatus for queuing medicines ina single file before they are introduced one-by-one to theidentification unit.

Alternatively or in addition, said image capture apparatus comprises atleast one digital camera having a lens, an aperture, a shutter, and anelectronic image sensor. Alternatively or in addition, the devicefurther includes a sorting apparatus for routing said target medicinefrom said identification unit to a desired location after the targetmedicine is processed or identified. Alternatively or in addition, saidsorting apparatus further comprise flapper elements controlled byelectric motors. Alternatively or in addition, said image identificationunit further comprises a release controlled by an electric motor to keepthe target medicine in the identification chamber longer for imageprocessing and identification and to release said target medicine intothe sorting apparatus once the image processing or identification iscomplete.

Additional features and advantages of the subject technology will be setforth in the description below, and in part will be apparent from thedescription, or may be learned by practice of the subject technology.The advantages of the subject technology will be realized and attainedby the structure particularly pointed out in the written description andclaims hereof as well as the appended drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide furtherunderstanding of the subject technology and are incorporated in andconstitute a part of this specification, illustrate aspects of thesubject technology and together with the description serve to explainthe principles of the subject technology. Like reference numbersindicate like elements. Furthermore, this specification is bestunderstood when read in conjunction with the included figures, whichdisclose one or more exemplary embodiments of an image intensifier. Inaccordance with standard practices, various features are not drawn toscale and are used for illustration purposes only.

FIG. 1 shows a schematic diagram of an exemplary medicine identifyingand/or sorting system of the subject technology.

FIG. 2 shows a flow diagram highlighting the high level steps of thepresent method and system.

FIG. 3 shows an embodiment of the subject technology shown in aschematic diagram, whereby a sorting device operates based on input fromthe present system and method.

FIG. 4 shows the sorting device output steps based on the matchingcategory provided by the system.

FIG. 5 shows a flow diagram of the present medicine matching method.

FIG. 6 shows the algorithm used to develop a strong match between atarget medicine being analyzed and a reference medicine image.

FIG. 7 illustrates an example block diagram of an image retrieval systemin accordance an exemplary embodiment of the subject technology.

FIG. 8 illustrates an example block diagram of a characterizer thatprovides an index to indexed lists of image identifiers in accordancewith an exemplary embodiment of the subject technology.

FIG. 9 illustrates an example flow chart for characterizing an image forentry in indexed lists of image identifiers in accordance with anexemplary embodiment of the subject technology.

FIG. 10 illustrates an example flow chart for retrieving images that aresimilar to a target image in accordance with an exemplary embodiment ofthe subject technology.

FIG. 11A shows a view of an exemplary embodiment of the subjecttechnology deployed in a sorting device, whereby a plurality ofmedicines can be loaded and individually analyzed and thereafter sorted.

FIG. 11B. is a perspective view of another alternative embodiment of alinear medicine distribution manifold for a medicine processing machineaccording to an embodiment of the present subject technology.

FIG. 12a shows a view of an exemplary embodiment of the present systemand method, whereby a handheld device is deployed to capture and processthe medicine image.

FIG. 12b shows a view of an exemplary embodiment of the present systemand method, whereby a handheld device is deployed to capture and processthe medicine image using access to a remote server.

FIG. 13 is a pictorial view of a configuration of an exemplaryembodiment of the subject technology in which a sorting and queuingapparatus is shown.

FIG. 14 is another rendition of the sorting and queuing apparatus inaccordance with an exemplary embodiment of the subject technology.

FIG. 15A is an exemplary bottom or underside view of an exemplary guideplate of the subject technology, which controls the movement ofmedicines before they are introduced to the medicine identificationchamber or unit.

FIG. 15B is an exemplary illustration showing section A of guide plate1022 depicted in FIG. 15A.

FIG. 16 is a pictorial view showing a ramp and window portion of theguide plate of FIG. 15A with an added illustration of the movement ofmedicine AB as it is moving on pad 1012 and being ejected from the guideplate by one of the ramps 1076.

FIG. 17A shows various exemplary ramps of the guide plate of FIG. 15A

FIG. 17B is a top view of the periphery of an exemplary sorter of thesubject technology, in which four exemplary ramps which face the guideplate windows are shown.

FIG. 18 is an exemplary side view of the medicine processing machine ofthe present disclosure with its various modules.

FIG. 19 illustrates an exemplary medicine distribution apparatus andcollection bins of the medicine-processing machine which relies on aplurality of diverters to divert medicines between specific collectionbins.

FIG. 20 illustrates an exemplary medicine identification and sortingdevice or system for receiving, queuing, identifying and sorting mixedmedicines.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the subject technology asclaimed.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth to provide a full understanding of the subject technology. It willbe apparent, however, to one ordinarily skilled in the art that thesubject technology may be practiced without some of these specificdetails. In other instances, well-known structures and techniques havenot been shown in detail so as not to obscure the subject technology.

To facilitate an understanding of the present subject technology, anumber of terms and phrases are defined below:

Definitions

A phrase such as “an embodiment” does not imply that such embodiment isessential to the subject technology or that such embodiment applies toall configurations of the subject technology. A disclosure relating toan embodiment may apply to all embodiments, or one or more embodiments.An embodiment may provide one or more examples of the disclosure. Aphrase such “an embodiment” may refer to one or more embodiments andvice versa. A phrase such as “a configuration” does not imply that suchconfiguration is essential to the subject technology or that suchconfiguration applies to all configurations of the subject technology. Adisclosure relating to a configuration may apply to all configurations,or one or more configurations. A configuration may provide one or moreexamples of the disclosure. A phrase such as “a configuration” may referto one or more configurations and vice versa.

Furthermore, to the extent that the term “include,” “have,” or the likeis used in the description or the claims, such term is intended to beinclusive in a manner similar to the term “comprise” as “comprise” isinterpreted when employed as a transitional word in a claim.

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any embodiment described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments.

A reference to an element in the singular is not intended to mean “oneand only one” unless specifically stated, but rather “one or more.”Underlined, bold and/or italicized headings and subheadings are used forconvenience only, do not limit the subject technology, and are notreferred to in connection with the interpretation of the description ofthe subject technology. All structural and functional equivalents to theelements of the various configurations described throughout thisdisclosure that are known or later come to be known to those of ordinaryskill in the art are expressly incorporated herein by reference andintended to be encompassed by the subject technology. Moreover, nothingdisclosed herein is intended to be dedicated to the public regardless ofwhether such disclosure is explicitly recited in the above description.

Unless otherwise indicated, all numbers expressing quantities such asflow volume or flow rate and so forth as used in the specification andclaims are to be understood as being modified in all instances by theterm “about.” The term “about” as used herein in reference toquantitative measurements not including the measurement of the mass of acompound, refers to the indicated value plus or minus 10%.

As used herein, the term “pill” refers to a wide variety of solid orsemi-solid oral dosage forms used for delivering one or more compoundsto a subject. The term “pill” includes, but is not limited to tablets ofany shapes (e.g., round, oblong, oval, square, rectangle, diamond,3-sided, 5-sided, 6-sided, 7-sided, 8-sided, heart shape, donut shape,or other shapes), capsules, caplets, gel caps, lozenges and the like.

Although the subject technology has been described with reference topills, one of ordinary skill in the art recognizes that the method ofthe subject technology is not limited to sorting and identifying pillsonly. The method of the subject technology is equally applicable tosorting and identifying any pharmaceutical dosage form includingmedicines that are in liquid form and are bottled (e.g., syrups orampules) or are for administration to an individual through routes otherthan an oral (e.g., suppositories). Therefore, as used herein the term“medicine” refers to any pharmaceutical dosage form that has at leastone unique physical feature (e.g., shape, color, surface lines,imprints, markings, deboss, emboss, groove, writing, label, and otherphysical indicia) that can be used for identification and sorting by thesystem or device of the subject technology and in accordance to themethod described herein. Thus, the term “medicine” includes anypharmaceutical dosage form that is identifiable by the method and deviceof the subject technology and includes, but is not limited to, tabletsof any shapes (e.g., round, oblong, oval, square, rectangle, diamond,3-sided, 5-sided, 6-sided, 7-sided, 8-sided, heart shape, donut shape,or other shapes), capsules, caplets, gel caps, lozenges, liquidmedicines (in their original bottles), ampules, suppositories, inhalersand the like.

Reference is made herein to the attached drawings. Like referencenumerals are used throughout the drawings to depict like or similarelements of the image processing medicine discriminator. For thepurposes of presenting a brief and clear description of the subjecttechnology, the preferred embodiment will be discussed as used foridentifying and sorting medicines based on an image processing methodand a system. The figures are intended for representative purposes onlyand should not be considered to be limiting (to e.g., pills) in anyrespect. For example, while the figures may show the device and systemof the subject technology is configured to sort pills, the teachings ofthe subject technology is equally applicable to sorting otherpharmaceutical dosage forms including liquid dosage forms (e.g., syrups,ampules) or aerosol dosage forms (e.g., inhalers), which are in theiroriginal bottles or containers.

The subject technology describes in terms of a method and system ordevice for analyzing a target medicine using an image capturing meansand an image processing means, whereby the digital image(s) of a targetmedicine is taken or created by the image capturing device and, by theimage processing device, said digital image is compared to referencemedicine images stored within a database to determine the identity ofthe target medicine. The system is deployed within a medicine sortingdevice or via a handheld image capture and processing device. Themedicine sorting embodiment of the system accepts several medicines andanalyzes each separately, while providing output in the form of asorting process or direct communication with the user. The handheldsystem employs a handheld electronic device (e.g. a smartphone device)that includes a camera of sufficient fidelity and a processing means foranalyzing and comparing the target medicine image to reference images.The reference images are stored locally on the device within a storagemeans, or alternatively the reference images are stored on a remoteserver, whereby the handheld device or sorting embodiment has thecapability of communicating with the remote server via a networkinterface means (e.g. a wireless antenna chip or Ethernet port). Themethod deployed for analyzing the medicines utilizes an image processingunit having several line and surface algorithms, whereby the details ofthe medicine surface characteristics and the shape or geometry of themedicine is compared to a plurality of reference medicine images withina retrievable database, whereby the success of the match is givenprobability as to an absolute match between the target medicine and areference medicine image. The match probability using several differentmatching algorithms are compared with one another to determine thehighest probability match before communicating to the user the identityof the medicine (if a match is indeed available in the referencemedicine database and if the target medicine can indeed be analyzedgiven its surface properties). In some embodiments, the success of thematch between the image of the target medicine and the images of thereference medicines is given as percent identity. For example, a percentidentity of 70% or more indicates a match. In some embodiments a percentidentity of 80% or more, 85% or more, 90% or more, 95% or more, or 98%or more, indicates a match between the target medicine and the referencemedicine, which results in identification of the target medicine.

Referring now to FIG. 1, there is shown a schematic diagram of theelements of the present system, whereby the system is capable ofanalyzing a target medicine 12 based on an image processing method andcomparing the processed image 14 to a stored database of known medicineimages. The system comprises an image capture unit or apparatus 11, animage processing unit 21, and a storage means 31, whereby the elementsfunction to provide an image processing computer system that provides amedicine match output 41 based on the results of the processing method.The image capturing means 11 preferably comprises at least one digitalcamera 13 or suitable image capturing technology that is capable ofcreating a digital image 14 of a target medicine 12 or a portionthereof, of sufficient fidelity such that the target medicine's surfacecharacteristics such as imprints, lines, contours, colors, markings,geometry, and texture can be seen with clarity for further processing.The camera 13 further includes elements commonly found in the art ofdigital image capturing devices, including a lens, an aperture, ashutter, an electronic image sensor, and an illuminating flash. Multiplecameras 13 may be deployed to simultaneously capture an image 14 ofvarious sides of the medicine for processing and surface recognition andfor improved matching. In an embodiment, the camera(s) 13 are installedat various locations in the system of the subject technology and arecapable of imaging a medicine as various locations and from differentangles.

The image processing unit 21 comprises a processing means 22 such as amicroprocessor or central processing unit (CPU) 22 and a main memory 23.The processing means 22 carries out programmed instructions of thematching method and carries out the operational instructions for thesystem elements. A storage means 31 stores digital information relatedto the reference images and the processing instructions for which theprocessing means 22 to carry out. A remote storage means 31 may also beutilized to retain the reference image information, thus reducing thelocal storage capacity requirements and allowing for updates to thedatabase of images and the system to quickly be uploaded or changedwithout uploading new information to the local storage 31. The localstorage means 31 comprises a mass storage device such as a computer harddisk or removable media, while the remote storage means may comprise ahard disk or server accessed remotely through a network accessed using anetwork interface means such as a wireless antenna chip or Ethernetport. In an embodiment, the storage means 31 is a cloud database. Inanother embodiment, the storage means 31 is a national or internationalmedicine registry database containing detailed data about all physicaland chemical attributes and characteristics of any and all medicines(including illicit drugs) produced anywhere in the world. The databasefurther includes drug interaction data, poison control and preventioninformation, and any other information that may be of use to physicians,nurses, emergency responders, law enforcement officers, soldiers oranyone who may be interested learning about the identity or use of amedicine. This database or registry is continuously updated byinformation received from different agencies (e.g., Food and DrugAdministration, Environmental Protection Agency, Drug EnforcementAgency, Customs and Border Protection Agency, and like agencies fromaround the world), hospitals, organizations and drug companies.

In operation, the image of a target medicine 12 is first captured usingthe image capture unit or apparatus 11, whereby at least a portion orone side of the medicine 12 is captured. Thereafter, the imageprocessing unit or apparatus 21 interrogates the target image 14 andmakes modifications thereto to highlight its shape, color, surfacelines, imprints, markings, deboss, emboss, groove, and other physicalindicia or characteristics. The processing means 21 then compares thetarget image 14 with reference images within the storage means 31 in aniterative process to determine an appropriate match. Several matchingcriteria are used, where after the medicine 12 is classified into, forexample, one of three output categories 41: the target medicinedecidedly matched with a referenced medicine image, the target medicinebeing unmatched, or the target medicine match being undetermined basedon the quality of the target medicine, its captured image, or based onthe limited extent of the reference image database. The output 41 isprovided to a user in a plurality of ways, including a visual indicationof the match output or by sorting the target medicine 12 based on acategory (e.g., match/unmatched/undetermined). In case of a match,additional information may be displayed to the user. This informationincludes, for example, identity of the target medicine, the therapeuticcategory of the medicine, whether the medicine is a controlled substanceor not, what should be done in case of overdose of the medicine, orvarious other information that may of use to the user. In an embodiment,the output 41 is linked to a sorting mechanism that will allow for thematched medicines to be routed to and sorted in specific bins orcontainers.

Referring now to FIG. 2, there is shown a flow diagram outlining anexemplary method for carrying out the image processing of the subjecttechnology. According to this exemplary configuration, a target medicineis first interrogated for its features and then identified based onreference medicine images. The method initiates when a target medicineimage, or a portion thereof, is acquired 101 by the image capture meanssuch as a camera. This forms an input image to be analyzed, modified,and then compared with reference images in the system. The input imageis saved 102 to the storage means and accessed using the main memory ofthe processing means. Thereafter, the main memory of the processingmeans loads consecutive references images 103 from the storage means tobe compared separately with the target image. The target image is thenanalyzed using a matching algorithm and compared 104 with each referenceimage, loaded consecutively. The geometry of the medicine, the lines ofthe medicine surface, the grooves (if any) on the medicine, the color(s)of the medicine, or other physical attributes of the medicine are allcompared against the loaded reference image to determine a probablematch.

The matching process is an iterative process by which several differentmatching algorithms are deployed to determine the algorithm thatprovides the highest probability of match over a confidence interval. Ifa suitable match threshold is not surpassed, consecutive referenceimages are cycled to determine a more appropriate and higher probabilitymatch. If the highest match probability does not meet a suitablethreshold, the output 105 is revealed as unmatched and the medicineidentity is not claimed. If the matching probability is sufficiency lowor if the processing means cannot find suitable features on the targetmedicine to match, the output 105 is shown as undetermined. Finally, ifa suitable match is found, the medicine identity is revealed 106 to theuser or used in the system to sort the medicine appropriately. Thisprocess utilizes the image processing means to cycle through matchingalgorithms and the reference images to match the target medicine withsufficient certainty. In an embodiment, the fidelity of the imagecapture means, therefore, the robustness of the image processing means,and the comprehensiveness of the reference image database are suitableto provide accurate output results, while also preventing falsepositives or false negatives.

Referring now to FIG. 3, there is shown a schematic and exemplary viewof the subject technology, wherein the elements of the medicine sortingembodiment are shown. The present method, when deployed in a medicineseparating apparatus, employs the same aforementioned system elements,however the output 41 involves sorting the target medicine based on theresults of the image processing and the matching results. In thisembodiment, at least one camera 13 is present within the system tofunction as the image capture unit or apparatus 11. The image processingunit or apparatus 21 functions as a means to processing the targetmedicine image, the image capture unit controller, and the controller ofthe sorting apparatus 41 after the medicine match probability has beendetermined. The image processing unit or apparatus 21 comprises theprocessing means 22 and main memory 23, which draws instructions from astorage means 31 to process the captured image of the target medicineand to compare the captured image to reference images also stored on thestorage means 31. As mentioned, the reference images may also be storedremotely on a secondary storage means to reduce local storagerequirements and to improve efficiency of updating the system. Based onthe results of the image processing, the match output 41 comprisesreleasing the target medicine from an identification chamber to asorting pathway or chamber. The medicine travels to a sorting pathwayand is routed to a specific bin or collection chamber. It iscontemplated that electric motors or stepper motors may be utilized inthe identification chamber to facilitate the identification the medicine(by slowing or stopping the medicine for imaging) and also to direct themedicine through a particular pathway 201 using flappers or othersuitable structures. The exact design and structural elements deployedwithin the sorting device can vary depending on the desired application,desired output, and the user requirements; however the basic systemelements are retained prior to the sorting process. In an embodiment, amedicine travels through a plurality of image processing units and/orsorting pathways to be finally sorted in a desired storage bin.

In an embodiment, to facilitate passage of the medicine 12 through theidentification chamber 202, the chamber is free of electric or steppermotors designed to stop the medicines as their images are captured.Instead, in some embodiments, the chamber includes medicine stops.Medicine stops are obstructions within the identification chamber 202that provide a means for slowing or temporarily stopping the medicine 12as the medicines travels through the identification chamber 202. Thiswill minimize any blurring of the medicine image. It will also allow forthe system to have more time to identify the medicine as the medicineapproaches or enters the medicine pathway to be routed to a specific binor collection chamber.

A medicine stop may be an obstruction (e.g., one or more flaps, or thelike) created that disrupts the natural traveling rate of the medicine12 through the identification chamber 202. An obstruction may be anydeviation from the smooth surface of the wall in the identificationchamber 202. By way of example only, a flap, bump or the like may beplaced within the identification chamber at or near the regions wherethe camera(s) 13 or imaging devices are installed. A medicine 12traveling over the flap or bump will naturally slow down or stop for afraction of second. In some embodiments, friction-creating protrusions,such as brushes, may project into the medicine path to slow down therolling or falling medicine through the identification chamber 202. Ifthe obstruction is placed within the identification chamber 202, theimage capture device 13 can capture the image of the medicine as itslows down or stops, allowing for a clear shot. In embodiments utilizingfriction creating obstructions, such as bumps, protrusions, brushes, andthe like, the obstructions may be adjustable so that if the medicine isstopped, the obstruction can be moved out from the pathway of themedicine to allow the medicine to resume forward or downward towards thesorting pathway and collection bins.

In some embodiments, no obstructions or medicine stops are utilized. Inother embodiments, the medicine may be transported through the imagecapturing device and image processing device by a conveyor belt. Theimaging device(s) 13 may be high speed cameras that can capture a clearimage of a moving medicine 12. Furthermore, the speed of the medicine 12may be adjusted by adjusting the angle of the identification chamberrelative to the ground to slow the medicine 12 down as necessarydepending on the quality of the imaging device(s) 13.

In some embodiments, a trigger may be set up to time the image capturingprocess to acquire an image just as the medicine 12 passes in front ofthe image capturing device(s) 13. For example, a beam of light may bedirected transversally through, or onto, the wall within the path of themedicine 12. When the medicine 12 passes through the beam of light todisrupt the transmission of the light, a signal can be sent to thecamera(s) 13 to acquire the image immediately or within a specifiedtime. A similar trigger can be set up for, or shared with, a second,third, fourth, and so on, camera(s) 13.

To assure the imaging device(s) 13 can capture the entire image or alarge portion of the image of the medicine 12, the image field may bebroad or that several imaging device (3) capture the target medicineimage from different angles. In some embodiments, once the trigger isactuated, the imaging device(s) 13 can begin capturing a series ofimages in rapid succession for a period of time. Alternatively, a stoptrigger can be positioned downstream of the image capture device suchthat actuation of the stop trigger stops the image capturing process.The stop trigger, like the acquisition trigger, may be a beam of light,disruption of which causes the image capture device to stop takingpictures. In another embodiment, the imaging device(s) 13 continuouslyacquire images.

In some embodiments, lighting can be installed within the identificationchamber 202 at or near the imaging device(s) 13 for providing sufficientlighting for capturing better quality images of the target medicine 12.The lighting consists of a plurality of lighting elements affixed in andaround the medicine pathway through the identification chamber 202, forexample, in or around the flaps within the identification chamber 202,such that medicines in this region are illuminated.

The emission source for the lighting elements may be fluorescent,halogen, xenon gas, light emitting diode (“LED”) or the like. In oneembodiment, the lighting elements are high current, high intensity,flash-LEDs, due to their longevity, physically robust design, and lowheat dissipation. The lighting elements may be affixed to the imageidentification chamber by solder, glue, epoxy, mechanical fasteners, orthe like. The electrical leads of the lighting elements may be connectedin series, parallel or some combination, to an external power source,and/or triggering source, via wires.

Referring now to FIG. 4, there is shown a flow diagram related to anexemplary embodiment of the output 41 of the medicine sorting embodimentof the present system. In this embodiment, after the target medicinematch category 105 has been determined, the processing meanscommunicates with the electric motor controllers or stepper motorcontrollers to toggle at least one sorting means 201 within the deviceto direct the target medicine to a specific location based on its matchcategory 105. In one embodiment, flappers are utilized to direct thetarget medicine from its identification chamber and into a prescribedbin. In an embodiment where the target medicine is first supportedwithin the identification chamber, once the match category 105 has beendetermined and the flappers have been toggled 201 to the correct outputbin, a release within the target medicine identification chamber 202 istoggled to drop or flow the target medicine to the output bin. Thisnotifies the user whether the medicine is matched, or within whatcategory the specific medicine can be classified. In other embodiments,a visual output may also accompany the physical sorting process suchthat the user is notified of the medicine match category and/orinformation about the medicine. In another embodiment, the user selectswhere or which bin a medicine of a specific characteristic (e.g., painkillers, antibiotics) is to be sorted in.

Referring now to FIG. 5, there is shown a flow diagram outlining anexemplary medicine image processing method according to the subjecttechnology. The first step in the process involves dropping or sending atarget medicine through an identification chamber or isolating anindividual medicine therein to be processed, after which at least oneimage of the medicine is captured. The target medicine image forms aninput image for the process that is first loaded 301. While the targetmedicine image is loaded, the first reference medicine image is loaded302 from within the reference image database. From here, the input imageis adjusted 303 to account for an image of the medicine that is notperfectly parallel to the image capture unit lens. Depending on thecircumstances with which the input image is taken, the medicine can betilted and thus skew the image thereof, creating an illusion of adeformed or skewed medicine shape and corresponding surface lines. Tocompensate for a skewed input image, the method performs an ellipsetransform operation to project the skewed image onto a plane whosenormal is directed at the camera lens. Adjusting the input image foroff-vertical camera angles involves recognizing the boundary of theobject and if the boundary has an elliptical shape rather than acircular shape. The major and minor axes are then determined; whereafterthe transformation proceeds by determining the major and minor axes ofthe target medicine and stretching the image parallel to the minor axiswhile shrinking it parallel to the major axis until they are equal. Thismay also require some keystoning operations. The goal is to provide acorrected input image for proper comparison with reference images.

Once the input image has been transformed 303, the input image iscompared 304 with the loaded reference image. The input image is scaled305 uniformly to match the approximate size (diameter) of the referencemedicine image. To reduce noise and minor imperfections of the targetmedicine surface (and background area) within the input image, the inputimage is blurred 306 to soften the image for improved detection of itsmajor surface lines and edges. Once blurred, the edges on the inputimage are detected 307 using a Canny edge detector operation. Majoredges on the target medicine are thus detected and thereafter marked 308on the input image for comparison to the reference image. After edgesare detected, surface lines of the input image are detected using a linetransform operation (Standard Hough Line Transform or Probabilistic LineTransform). These lines are also marked on the input image 310,modifying the input image based first on detected edges and then basedon detected lines. The input image is modified and stored within thestorage means. Once edges and lines are marked on the input image, theinput image is overlaid onto the reference image 311 for the matchingprocedure to commence. Up to this step, the steps have involvedmodifying the input image such that the matching steps will proceed withgreater probability of match if indeed a match does exist within thereference image database.

Matching the modified input image with the loaded reference imageproceeds by using the detected edges and lines from the modified inputimage, rotating the modified input image, and sliding the modified inputimage to correspond with the lines and edges of the reference image.Either the reference image or the modified input image may be rotatedand slid during this comparison. The image being rotated and slid ismoved one pixel at time. At each pixel location, a metric is calculatedthat represents how “good” or “bad” the match is at that pixel location(or how similar the reference is to that particular area of the modifiedinput image). By rotating, the image being handled is rotated in asequence of five degrees from center, and repeating the sliding process.This rotation can be repeated for a full rotation (360 degrees) of theimage being handled (the modified input image or the reference image).For each pixel location during the rotation and sliding operation, themetric is stored in a results matrix (R). Each location (x,y) in Rcontains the match metric.

The process of matching the modified input image with the loadedreference image 312 proceeds using several different algorithms todetermine the highest match probability. The highest match probabilityis then utilized as the result of the matching operation 312. Thesepercentages can change dynamically because of the lighting on the targetmedicine surfaces and the wear of the medicine. The following is a listof algorithms utilized in the matching procedure. Each matchingalgorithm is well known in the art of image processing. The matchingprocess proceeds by finding areas of an input image matching thetemplate image whereby an each of the following algorithms is utilized:

Squared Difference Algorithm²:

${R\left( {x,y} \right)} = {\sum\limits_{x^{\prime},y^{~\prime}}\mspace{14mu}\left( {{T\left( {x^{\prime},y^{\prime}} \right)} - {I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}} \right)^{2}}$

Normalized Squared Difference Algorithm:

${R\left( {x,y} \right)} = \frac{\Sigma_{x^{\prime},y^{\prime}}\mspace{14mu}\left( {{T\left( {x^{\prime},y^{\prime}} \right)} - {I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}} \right)^{2}}{\sqrt{\Sigma_{x^{\prime},y^{\prime}}\mspace{14mu}{{T\left( {x^{\prime},y^{\prime}} \right)}^{2} \cdot \Sigma_{x^{\prime},y^{\prime}}}{I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}^{2}}}$

Cross Correlation Algorithm:

${R\left( {x,y} \right)} = {\sum\limits_{x^{\prime},y^{~\prime}}\mspace{14mu}\left( {{T\left( {x^{\prime},y^{\prime}} \right)} \cdot {I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}} \right)}$

Normalized Cross Correlation Algorithm:

${R\left( {x,y} \right)} = \frac{\Sigma_{x^{\prime},y^{\prime}}\left( {{T\left( {x^{\prime},y^{\prime}} \right)} \cdot {I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}} \right)}{\sqrt{\Sigma_{x^{\prime},y^{\prime}}\mspace{14mu}{{T\left( {x^{\prime},y^{\prime}} \right)}^{2} \cdot \Sigma_{x^{\prime},y^{\prime}}}{I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}^{2}}}$

Correlation Coefficient Algorithm:

${R\left( {x,y} \right)} = {\sum\limits_{x^{\prime},y^{~\prime}}\left( {{T\left( {x^{\prime},y^{\prime}} \right)} \cdot {I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}} \right)}$where:                                        ${T\left( {x^{\prime},y^{\prime}} \right)} = {{T\left( {x^{\prime},y^{\prime}} \right)} - {1\text{/}{\left( {w \cdot h} \right) \cdot {\sum\limits_{x^{''},y^{''}}{T\left( {x^{''},y^{''}} \right)}}}}}$${T\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)} = {{I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)} - {1\text{/}{\left( {w \cdot h} \right) \cdot {\sum\limits_{x^{''},y^{''}}{I\left( {{x + x^{''}},{y + y^{''}}} \right)}}}}}$

Normalized Correlation Coefficient Algorithm:

${R\left( {x,y} \right)} = \frac{\Sigma_{x^{\prime},y^{\prime}}\left( {{T\left( {x^{\prime},y^{\prime}} \right)} \cdot {I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}} \right)}{\sqrt{\Sigma_{x^{\prime},y^{\prime}}\mspace{14mu}{{T\left( {x^{\prime},y^{\prime}} \right)}^{2} \cdot \Sigma_{x^{\prime},y^{\prime}}}{I\left( {{x + x^{\prime}},{y + y^{\prime}}} \right)}^{2}}}$

The foregoing algorithms are utilized to match the modified input imageto the reference image once overlaid over one another. All of thesealgorithms can be cycled through individually or some combinationthereof. The highest match probability resulting from the deployedalgorithms are then utilized as the match result for output of thesystem.

Referring now to FIG. 6, there is shown an exemplary expanded flowdiagram of the matching procedure 312, whereby the matching methods nthrough N are executed 401 in an iterative process. The given matchingalgorithm derives a match probability 402, which is then utilized in theoverall flow of FIG. 5 to compare to the algorithm's match probablyagainst preceding match probabilities to determine which algorithm hasprovided the highest match probability. Referring back to FIG. 5, thematch output from the given algorithm is normalized 313 before regionsof the input image with higher matching probability are identified 314for matching surface marks and indicia of the medicine. The identifyingmarks or indicia are then recognized 315 using, for example, opticalcharacter recognition (OCR) to determine what the indicia read. Once thematching procedure is conducted, the probably of the match is thencompared against any previous algorithm's probability to determine ifthe probably of the given iteration is acceptable 316. If the matchingalgorithms have not been exhausted and if the matching probably is notto a sufficient standard, the matching process 312 initiates again witha different matching algorithm, whereby its match probability isdetermined. This proceeds until a sufficiently high probably of match isdetermined or if the algorithms have been exhausted.

If the results indeed provide a match with a sufficient probability, theresults of the match being relayed 319 to the user or utilized in thesorting process. If the match probability is not sufficiently high, asubsequent reference image is loaded 302 for matching another referencemedicine to the input image. If the reference images within the databasehave been exhausted, the results of the non-match are relayed to theuser 319 or forwarded to the sorting process for appropriate action. Inthis way, the input image is matched using several different matchingmethods for each reference image until a satisfactory output is reached.This is a sample flow that fulfills the goals of the present imageprocessing method. It is contemplated that departures or more efficientsteps may be incorporated in later designs of the method, however thebasis of the medicine identification procedure is image processing usingline and edge detection and probabilistic matching algorithms that scaneach pixel of the input image against the reference image and identifyphysical characteristics or indicia on the target medicine surface.

In some embodiments, to provide for efficiency in the comparisonprocess, indexed lists of image identifiers (such as characterizationsbased upon the color content or edge content of partitions of an image)are maintained, and the count of similar characterizations of an imageis determined by the count of occurrences of the image's identifier inselected lists. The selected lists are determined by a characterizationof a target image from which similar images are to be identified. Forexample, the indexing and retrieval techniques disclosed in U.S. Pat.No. 6,253,201 (hereby incorporated herein by reference) are suitable foruse in the subject technology. The indexing method allows for quickimage retrieval and comparison, which leads to a faster or real timeidentification of the medicines as they travel through theidentification chamber 202. Multiple indexes can be associated with oneor more characteristic measures of each partition, allowing for imageretrieval based on one or more characteristics of the target image.

FIG. 7 illustrates an example block diagram of an exemplary indexing andimage retrieval system. The image retrieval system includes acharacterizer 520 that produces indexes 502, 512 to lists of imageidentifiers 530, and a search engine 540 that processes selected listsof image identifiers 535 to determine the images 581 that have a highnumber of occurrences 561 in the selected lists 535.

By introducing a medicine to the medicine identification system of thesubject technology, a target medicine image 501 is provided to the imageretrieval system of FIG. 7 to determine the reference medicine images511 of a reference medicine image database 510 that are similar incharacteristics to the target medicine image 501. The source 500 of thetarget medicine image 501 may be a digitizer, a camera, and the like.

The reference database of images 510, which is similar to the referenceimage storage database 31 shown in previous figures, may be located in alocal or remote disk storage, a memory device, a central server, anonline server, a cloud computing site, and the like. The referencemedicine images are created and stored using input devices such asscanners, digitizers, and cameras, as discussed above. Additionally,they could be a series of related images as might be found in an MPEGencoded video, or a conventional video program. The term database inthis context means a collection of items (images), each of which can beuniquely identified. As is known in the art, a database may bedistributed, and need not reside in a single physical device, nor needthe addressing scheme be common among all devices. That is, as usedherein, the reference medicine image database 510 is independent of thephysical media that contains the images, and independent of the mediadependent techniques for accessing each image. Each image 511 in thereference medicine image database 510 is provided to the characterizer520 to create the indexed lists of image identifiers 530.

FIG. 8 illustrates an exemplary block diagram of a characterizer 520that provides an index 602 to the indexed lists of image identifiers530. The characterizer 520 includes a partitioner 610, a characteristicprocessor 620, and a quantizer 630. The partitioner 610 partitions animage 601 into an array of partitions; typically the array is a 4×4,8×8, or 16×16 partitioning of the image. The index 602 includes anidentification P of each partition, and an indexed characterization Idxthat characterizes the partition as one of a set of predefined indexedcharacterizations. The characteristic processor 620 processes eachpartition P based on the characteristic used to describe an image andproduces a characteristic measure 621 that describes the partition. Ingeneral, the characteristic measure 621 is a histogram of theoccurrences of the components of the descriptive characteristic, forexample, the number of occurrences of particular colors, or the numberof occurrences of particular types of edges. Other techniques are knownin the art for deriving a characteristic measure or set of measures thatdescribe an image, such as combinations of particular shapes, etc. Thequantizer 630 transforms the characteristic measure 621 that is producedby the characteristic processor 620 into one of a plurality ofpredefined indexed characterizations Idx. In the most straightforwardexample, the quantizer 630 transforms a histogram of occurrences of thecomponents of the descriptive characteristic into a set of proportionsof each component contained in each partition, and then quantizes eachproportion into predefined ‘bins’ such as quartiles, octiles, etc. Inthe general case, each of the predefined indexed characterizations Idxis associated with a location in the parameter space of thecharacteristic measure 621, and a region about this location. Thequantizer 630 determines the predefined indexed characterization Idxbased on the region in which the characteristic measure 621 lies. Thequantization provided by the quantizer 630 need not be uniform. Forexample, more indexed characterizations may be located in the area ofthe parameter space that corresponds to frequently occurring values ofthe characteristic measure, thereby providing for a greater degree ofdistinction among such values as compared to a uniform distribution ofthe indexed characterizations about the parameter space. The locationsof the indexed characterizations are typically called the quantizationlevels, or quantization centers; for example, in a colorcharacterization, the colors at the location of the indexedcharacterizations are termed the color centers. The quantization neednot be un-valued. For example, multiple indexed characterizations may beprovided for each characteristic measure 621, as will be discussedbelow.

Each index 602 provided by the characterizer 520 is used to store anidentifier 615 corresponding to the image 601. Typically, the identifier615 is a unique numerical value for each image 601, and this numericalvalue corresponds to a list of locations (not shown) that identify wherethe image 601 is located. For example, the location specified in thelist corresponding to the identifier 615 may be a conventional computerpath name that identifies a file that contains the image 601.Alternatively, the location could be text that is imprinted on amedicine, followed by the shape of the medicine. In FIG. 8, uppercaseletters are used to represent the particular image identifiers 615.

The identifier 615 of the image 601 is stored in each list 535 that isassociated with a partition P of the image that has an indexedcharacterization Idx. That is, for example, if partition P1 correspondsto the upper left corner of the images, and index I1 corresponds to anoccurrence of predominantly red and blue colors, then the list 535 awill be a list of the identifiers (A, D, Q, R, K) of all the images 511in the database 510 that have predominantly red and blue colors in theirupper left corner. List 535 b corresponds to the images 511 in thedatabase 510 that have predominantly red and blue colors in the area ofpartition P2, which may be, for example, the lower left corner of theimages.

Multiple indexed characterizations may be provided for each partition.For example, characterization 14 could correspond to the occurrence ofpredominantly horizontal edges, or to a partition having an averagebrightness of 25 lumens, etc. In this manner, multiple characterizationsof each partition (color, edges, markings, debosses, grooves, luminance,etc.) of a target medicine image 501 can be used to place the imageidentifier 615 into multiple lists 535. The retrieval of images canthereby include retrievals based on the similarity of images 511 to oneor more particular characteristics of the target medicine image 501. Themultiple characterizations may be of differing characteristics, such ascolor and shape, or of the same characterization, for example, acharacterization of occurrences or intensities of each primary color. Byproviding multiple indexes of the same characterization allows, forexample, a search for images having red colors in their upper rightpartitions, independent of the other colors that may also be present inthe upper right partitions. Multiple indexes of differentcharacterizations allow, for example, a search for images havinghorizontal edges and green color in the upper right partitions.

FIG. 9 illustrates an exemplary flow chart for characterizing an imagefor entry in indexed lists of image identifiers. The medicine image, ora portion thereof, is acquired, in 710, and an image identifier ID isassigned, at 720. The image is partitioned, at 730, and each partitionis processed in the loop 740-749. The partition is characterized at 742to form one or more characteristic measures. Each characteristicmeasure, such as color or shape, is processed in the loop 750-759. At752, the indexed characterization corresponding to the determined valueof the characteristic measure at the partition is determined. The imageidentifier corresponding to the image is appended to the list of imageidentifiers having the same indexed characterization at the samepartition, at 754. Each characteristic measure for each partition of theimage is similarly processed, as indicated by the “next” blocks 759,749. Note that the flowchart of FIG. 9 may be applied independently foreach image that is being characterized for entry into the indexed listsof image identifiers 530. The only dependency is the availability ofspace in the indexed lists to append the image identifier ID. Techniquescommon in the art, such as dynamic linked lists, are used in thepreferred embodiment to maximize the likelihood of the space beingavailable to append the entry.

FIG. 10 illustrates an example flow chart for retrieving characterizedreference medicine images 511 that are similar to a target image 501.The similarity is determined by counting the number of occurrences ofeach reference image 511 that has a corresponding partition with thesame characteristics as the target image 501. At 800, the count of thenumber of occurrences of each image identifier ID is initialized tozero. The target medicine image 501 is obtained and partitioned, at810-820. Each partition of the target image is processed in the loop830-839. The partition p is characterized at 832, using the samecharacterization measures as had been used to characterize the referenceimages 511, or a subset of these characterization measures. That is, forexample, if the reference images 511 have been characterized by colorand edge characteristics, the target image 501 may be characterized at832 for color characteristics only, or edge characteristics only, orboth color and edge characteristics. In this manner, for example, colortarget medicine photos can be compared for both composition and color.At 834, an indexed characterization Idx is determined for eachcharacteristic measure, using the same quantization scheme used fordetermining the indexed characterizations of the reference images.

In an embodiment, multiple indexed characterizations Idx may bedetermined for each characteristic measure, to overcome quantizationanomalies. Quantization anomalies occur, for example, when two imageshave similar characteristic measure, but receive differing indexedcharacterizations because the characteristic measures lie near theboundary between two indexed characterizations and the measure of eachof the images lie on opposite sides of the boundary. At 834, multipleindexed characterizations are produced whenever the characteristicmeasure lies within a specified range of the boundary between indexedcharacterizations. Other algorithms for generating multiple indexedcharacterizations from a target characteristic measure, for example,associating overlapping quantization regions to each indexedcharacterization, may also be used.

Each indexed characterization Idx is processed in the loop 840-849. Foreach indexed characterization Idx of each partition P, the list of imageidentifiers associated with this partition index (P, Idx) is extractedfrom the indexed lists of image identifiers, at 842. As noted above, thelist of image identifiers at each index is a list of all the images inthe database 510 that have the same indexed characterization of thepartition. At 844, the count of each image identifier ID that iscontained in the extracted list corresponding to (P, Idx) isaccumulated. If multiple indexed characterizations are associated witheach partition P, this accumulation of counts is dependent upon whetherthe multiple characterizations are dependent or independent. Forexample, if the characterizations are independent, such as color andedge characteristics, an image identifier ID occurring in each of twolists (P, color-Idx) and (P, edge-Idx) accumulates two counts, therebyaccumulating a higher count than an image identifier that only occurs inone of these lists. If the characterizations are dependent, however,such as redundant quantizations used to avoid quantization anomalies, asdiscussed above, an image identifier ID that occurs in multipledependent-index lists accumulates a single count. The occurrence of animage identifier ID two dependent-index lists accumulates the same countregardless of whether it occurs in either or both of thesedependent-index lists. In effect, the set of image identifiersassociated with each partition is the union of the sets of imageidentifiers in each of the dependent-index lists associated with thepartition. After all partitions are processed via the loop 830-839, thecount variable that is associated with each image identifier containsthe number of times each image identifier occurred in the lists thatcorrespond to the indexed characterization of the partitions of thetarget image. That is, the count is correlated to the number of similarcharacteristics between the reference and target images.

At 850, the counts of the image identifiers are sorted, and the locationof those having the highest count, i.e. those having the highestsimilarity to the target image, are presented to the user, at 860. In apreferred embodiment, the images corresponding to the image identifiersare presented to the user as well. Thus, as can be seen, the subjecttechnology provides for a determination of those reference medicineimages in a database 510 that have the most similar characteristics tothe target medicine image 501, without requiring a direct comparison ofthe characteristics of the target image to each reference image.

Various other image retrieval and processing methods that can be used inthe system or method of the subject technology are disclosed in, forexample, U.S. Pat. Nos. 8,542,951; 8,503,777; 8,463,045; 7,840,081;7,602,976; and 7,369,685, which are hereby incorporated herein byreference.

In another embodiment, a classifier may be used to classify the queryimage into one of a plurality of classes based on the representation ofthe query image. The classifier may include a neural network, a Bayesianbelief network, support vector machines (SVMs), fuzzy logic, HiddenMarkov Model (HMM), or any combination thereof, etc. The classifier maybe trained on a subset of the representations of the images stored inthe image database.

Referring now to FIG. 11, there is shown an exemplary cross section viewof the medicine identification and sorting embodiment of the presentdisclosure. In this embodiment, the system utilizes a sorting means asan output based on the input image processing. In an embodiment, one ormore imaging device(s) or camera(s) 13 are positioned along varioussides of a medicine identification chamber 202 through which the targetmedicine travels (by e.g., gravity or by a conveyor belt) during imagecapture and processing. The medicine identification chamber 202 acceptsa single medicine at a time, whereby medicines can be dropped into amedicine deposit area 902 having sloping sidewalls 903 or a funnel shapeto direct a target medicine into the identification chamber forprocessing. In an embodiment, the target medicines are transported intothe identification chamber in a single file or one-by-one by gravity orby a conveyor belt. In an embodiment, the target medicine may besupported within the chamber 202 via a perch controlled by the chamberrelease motor (not shown), which rotates the perch from a closed to andopen position. When closed, a target medicine is supported on its edgefor image capture of its different sides. When open, the perch withdrawsfrom the interior of the chamber to allow the target medicine to dropinto a sorting area. As provided above, the identification chamber maybe free of electric or stepper motors designed to stop the medicines astheir images are captured. Instead, in some embodiments, the chamberincludes medicines stops. Thus, in another embodiment, the targetmedicine may be slowed down as it travels through the identificationchamber by the medicine stops 906. Medicine stops are obstructionswithin the identification chamber 202 that provide a means for slowingor temporarily stopping the medicine 12 when it enters the imagingregion(s) of the identification chamber 202. The medicine stop may be anobstruction (e.g., one or more flaps, or the like) created that disruptsthe natural traveling rate of the medicine 12 through the identificationchamber 202. In an embodiment, the entire or partial structure of theidentification chamber 202 is made of transparent plastic orthermoplastic such as Plexiglas® or Lexan®. In another embodiment, thetransparent surface in the identification chamber 202 is constructed ofscratch resistant, optical grade glass such as Corning® Gorilla® Glass.

Once the target medicine is identified, the medicine continues itstravel towards the sorting area where it will be distributed tocollection bin by a distribution mechanism. For example, as shown inFIG. 11A, the flapper elements 206 within the sorting area control thedirection of the target medicine after it has been identified, wherebythe medicine is routed to a desired collection bin 904 therebelow. It iscontemplated that the number of flapper elements 206 and design of thesorting area may take several different forms, depending on the needs ofthe user and the number of output categories desired. See for exampleFIG. 11B, where a linear distribution manifold is illustrated. Thelinear distribution manifold has an inlet 930 and a plurality of outlets933 a-e which are disposed above collection bins or another distributionmechanism. The manifold contains a chute 931 pivotally attached to ahousing 932 of the manifold. The chute pivots so that a medicineentering the chute at the first end 930 can be directed to any one ofthe outlets 933 a-e.

It is desired to show a functional embodiment that can be used toanalyze and optically analyze medicines deposited into the devicequickly and efficiently. The target medicines are placed within thedeposit area 902 one by one, or alternatively a sorting or queuing meanswhich can accept a plurality of medicines at once and convey themindividually into the identification chamber. This sorting or queuingmeans will be discussed in more details below, whereafter the medicinesare sorted into collection bins 904 for the user.

Referring now to FIGS. 12a and 12b , there is shown an embodiment of thepresent system in which a handheld electronic device or smartphonedevice 911 is employed as a means of capturing and processing a targetmedicine 12 image. In this embodiment, the handheld device 911incorporates the image processing unit 21 and the image capture unit ofthe system. Notably, a camera 13 along the device 911 allows a user tocapture an image of the target medicine 12, store the image onto thelocal storage means 31 within the device 912 while controlling theprocess through an application on the device interface 912. The imageprocessing unit 21 proceeds as with the sorting embodiment, wherein thetarget medicine is captured as an input image to be compared againstreference images stored within a database. The database may be storedlocally on the device storage means 31, or alternatively the device mayhave network 701 connectivity. With connectivity to a network 913 usinga network interface means (e.g. a wireless antenna chip or Ethernetport), the database can be stored on a remote server 914, reducing thestorage needs of the device 911 and relying on the remote server 914 tostore the files of the reference medicine images. The network maycomprise an internet network, a local area network, or a wirelessnetwork. A further embodiment and a variation to that configurationshown in FIG. 12b is the option for running the image processing unit 21on the remote server 914, as opposed to running the processing means 21locally on the handheld electronic device 911. This embodiment allowsthe processing of the medicine 12 to occur remotely, where improvedcomputing power may be employed over that installed in the handhelddevice 911. The image is therefore processed remotely and the results ofthe information are relayed back through the network 913 for thehandheld device to relay results to the user. This embodiment isparticularly useful to law enforcement officers, first responders orgenerally to anyone who may need to identify a medicine immediatelywithout much effort other than capturing the image of anot-readily-identifiable medicine with a smart phone or the like.

Medicine Data Display

Once a medicine is identified, the system of the subject technologyprovides visual notification of the medicine identity as well asadditional data that may be customized according to the user's need. Themedicine identity data and information can be visualized through ascreen (e.g., smart phone screen, a monitor, TV screen and the like). Inan embodiment, the user may customize the information that can bedisplayed by the system and method of the subject technology. Forexample, an emergency responder may customize the system or device ofthe subject technology such that any of the following information isdisplayed: the medicine image; the medicines active ingredient, i.e.,the drug contained with the medicine; the drug's major function, thedrugs possible side effect, what the steps that must be taken in case ofoverdose or abuse; and/or the drug's antidote. See Table 1 below:

TABLE 1 Drug Image Drug's Name Hydrocodone/Acetaminophen (Vicodin ®); RXor Prescription drug Active Agent/Ingredient Combination of:Acetaminophen, Hydrocodone Drug's class: Analgesic (Pain Reliever),opioid Drug Overdose Sign & Extreme sleepiness Symptoms Breathingproblems Small pupils -- the black circle in the colored part of youreye Drug Antidoe Oxygen gas: for better breathing; Naloxoane (Narcan ®):only if breathing is seriously compromized; Activated charcoal with alaxative to soak up drug that is still left in your stomach orintestines. Tylenol ® or aspirin: if overdose is suspected but noserious symptom exists Pregnancy Risk Category C (Risk cannot be ruledout) Drug's Possible Side Effects: Sedating, habit-forming, dizziness,nausea and vomiting, impaired thinking and function

On the other hand, a law enforcement officer may only be interested indetermining whether the medicine is illicit or not. See Table 2 below:

TABLE 2 Drug Image Drug's Name Ecstasy Illicit - Controlled SubstanceYES¹ ¹A positive ID may accompany other forms of alarms ornotifications.Sorting or Queuing System

As discussed above with reference to FIG. 11A, the target medicines areeither placed within the deposit area 902 manually and one-by-one, oralternatively a sorting or queuing means which can accept a plurality ofmedicines at once and convey them individually into the identificationchamber is used. An exemplary operation of the sorting and queuingsystem of the subject technology is described below.

Referring initially to FIG. 13, basically, the exemplary sorter 1000employs a resilient disc in the form of pad 1012 of a rigid or anelastomer construction rotated on and by a turntable 1014 driven bymotor 1016 via belt 1017. A hopper 1018 (shown as partially broken away)is positioned about an opening 1020 in guide plate 1022, and medicinesto be sorted are inserted through this hopper. Guide plate 1022 issupported, by means not shown, at a selected spacing (e.g., from about0.001 mm to 100 mm) with respect to pad 1012, typically about 0.02 toabout 30 mm. A centrally positioned hub 1024 extends through an opening(not shown) in pad 1012 and is conventionally secured as by a threadedconnection to turntable 1014. Hub 1024 has a tapered surface whichfunctions to direct medicines in an off-center direction so that therewill always be some centrifugal force tending to cause medicines to moveoutward toward guide plate 1022. In FIG. 14, another exemplary sorter1000 is show in which the pad 1012 is driven by motor 1016 located belowthe sorter. The guide plate 1022 is positioned above the pad and issupported by hinges which allow the easy opening and closing of thesorter. Another exemplary sorter is shown in FIG. 20.

Referring now additionally to FIG. 15, the underside of guide plate 1022is configured to guide medicines rotated by pad 1012 to move in thedirection of the arrows in a circular and then spiral path outwardwithin an inner positioned recess 1034 which overall is oval inconfiguration and has an inner guide, or guide edge 1030, which extendsaround it. The medicines are moved outward by centrifugal force; and aremoved in a path governed by tapered inner facing edge 1030 of recess1034, this recess having, in general, a depth on the order of 0.05 to 30millimeter or deeper than the thickest medicine to be sorted. Thus, themedicines are free on the top surface of recess 1034. The first part oftheir travel is generally circular and during it the medicines areformed in a single file.

At approximately point 1040, edge 1030 of recess 1034 transitions, in arecess portion 1044, from being circular to a spiral, and thereaftermedicines are moved outward, along edge 1042, by the combination ofcircular movement of pad 1012 and centrifugal force. Recess region 1044may be of the same depth as other portions of recess 1034 or slightlyshallower. Where it is necessary to provide reduced depth, there wouldbe a gradual transition or slight ramp downward between central portion1035 of recess 1034 and recess region or portion 1044 and downwardlybetween recess region 1044 and region 1067, with reference to acounterclockwise direction of FIG. 15. In some embodiments, there is noreduced depth anywhere in the portions 1035, 1044 or 1067; and they allwill have the same depth. In some embodiments, one or more circularbushes are installed along the path A between the recess portions 1044and 1067 to facilitate the movement of the medicines along this path, orto break any bottlenecking that may have occurred in the recess 1044portion, or to cause medicines to move individually and in a single filealong the path A. Particularly, where there is a possibility that onemedicine may be on top of another medicine as they move to the portion1044, the movement of the brush(es) 1050, which is either slower orfaster than the rotational movement of the pad 1012, will urge thepiggybacked medicines to disengage from one another and to move in asingle file and behind each other as they move through area A of FIG.15. Also, if a medicine is standing on its side as it enters the portion1044, the action of the brush(es) together with the inclined shape ofthe edge 1042 will urge the medicine to lay flat as it enters the area Aof FIG. 15. Medicines that will not enter the recess portion 1044 willbe re-circulated around pad 1012 until they get the chance to enter theportion 1044. Thus, recess portion 1044 also forms a restrictedpassageway for a single file of medicines. This passageway is formedbetween outward projection 1062 and edge 1064 of recess 1044.

As the pad 1012 rotates, medicines are moved outwardly into the recessportion 1044 and thereby move around the guide edge 1062 until they aremoved circularly beyond the recess portion 1044 of the recess 1034 wherethey are free to move outwardly by centrifugal force. Freely movingmedicines finally form a single file in the recess portion 1067 and arerotated by pad 1012 to a peripheral area from point 1069 to point 1079(i.e., area B in FIG. 15, panel A) containing a plurality of medicineejection ramps which are each distinctively configured to eject adiscrete diameter of medicine, the largest being ejected first. Frompoint 1069 to point 1079, medicines come in contact with upwardlyextending ramps 1076-1 to 1076-n (FIGS. 15-16) based on their maximumdiameter (in case of round tablets) or width (in case of capsules,caplets, oval, oblong or polygonal tablets) or simply the maximum widththey occupy on the pad 1012.

The ramps 1076-1 to 1076-n are disposed on the edge 1072 facing thewindows 1080 around the outer edge of the guide plate 1022. Linesdesignated as in FIG. 17 (or X1-X4 in FIG. 17B) indicates a maximumdiameter of the circular path at any given ramp (1076-1 to 1076-n) alongwhich medicines may progress. This width or diameter, X, graduallynarrows or tapers from the ramp 1076-1 to the ramp 1076-n. Due to thecentrifugal force being applied to the medicines, for example, amedicine AB (FIG. 16) is urged outward along edge 1042 to point 1069(FIG. 15A) where the medicine AB comes in between the outer edge of theguide plate 1022 and the first ramp 1076-1. Once the medicine AB arrivesat a position where its maximum diameter or width is larger than X, theforce exerted by the outer edge of the guide plate 1022 (which is equalbut opposite in direction to the centrifugal force generated by therotating pad 1012) urges the medicine AB to go over the ramp 1076 (e.g.,any of 1076-1 to 1076-n). This action causes the medicine to be liftedup from the pad 1012 and ejected out of the sorter 1000 through thewindows 1080 (FIGS. 14 and 16). If a medicine is not engaged with a rampbecause of its diameter is less than X_(n) of that ramp, the centrifugalforce that is applied to the medicine by the rotational movement of thepad 1012 is not sufficient to cause the medicine to be ejected out thewindows 1080 without having gone over the ramps.

If a medicine's diameter or width is less than X at a given ramp, themedicine will simply pass that ramp without going over it; and as suchit will not be ejected out of the window 1080. For example, if, whiletraveling from point 1069 to point 1079 (FIG. 15), a medicine such as around tablet happens to be sanding on its side while moving forward,this medicine will finally go over a ramp at which the thickness of thetable (as the table is sanding on its side) is larger than the X at thatramp. The windows 1080 have a low beveled edge 1081 at the bottom (FIG.16) which will prevent medicines to exit the sorter 1000 without beingfirst engaged with the ramps. In one embodiment, the guide plate has alow beveled edge all around the outer periphery. The guide plate 1022 isdesigned so that the largest medicine exits the sorter 1000 first andthe smallest medicine exits the sorter 1000 last. The diameter X at theramp 1076-n may be zero millimeter or close to zero mm so that anypowder or crushed medicine that may be on the pad 1012 will exit thewindow opposite to this last ramp. The window opposite to ramp 1076-n(the last ramp) therefore may not have a bottom edge to facilitate theexit of the powdered materials from this window. This action of the lastramp will assist in a continuous cleaning of the surface of the pad1012.

Referring to FIGS. 17A and B, in various exemplary embodiments, eachramp includes a blade projecting outwardly from the edge 1072 andextending upwardly away from the pad 1012 with a slope of variabledegree angles with respect to the pad 1012. In an exemplary embodiment,the ramp has a slope of 3 to 75 degrees with respect to the pad 1012.The ramp may be straight with a constant slope or may have curves withvariable slopes. The outward facing edge of the ramp or blade may extendslightly downwardly or upwardly in the direction of the window 1080 sothat a medicine that goes on the ramp is ejected in the direction of thewindow 1080. The width of the ramp is constant or may be increasing, orboth, from the point closest to the pad 1012 to the point farthest fromthe pad 1012 (FIG. 17A). All the ramps 1076-1 to 1076-n may have of thesame size and shape (FIG. 17B) or may be of different sizes. Forexample, the ramp 1076-1 may be larger in size from the other ramps, andthe subsequent ramps may progressively get smaller in size from thepoint 1069 to the point 1079.

In some embodiments, the X value at each ramp ranges from about 30 mm toabout 0 mm. In an embodiment, X1 (the X value at ramp 1076-1, FIG. 17B)is about 9 mm, X2 is about 8.5 mm, X3 about 7.5 mm, X4 is about 6.5 mm,X5 is about 6 mm, X6 is about 5 mm, X7 is about 4.5 mm, X8 is about 3.5mm, X9 is about 2.5 mm. In an aspect related to this embodiment, theramps 1076-1 to 1076-9 will be able to capture and eject the medicineslisted in Table 3 below.

width/ diameter X RX (Brand Name) Dosage (mm)¹ (mm) Vicoprofen ®  10 mg13.4 9 Vicoprofen ®   50 mg/1000 mg 11 9 Vicoprofen ® 200 mg 9.5 9Vicoprofen ® 140 mg 9.5 9 Vicoprofen ®  10 mg 9.5 9 Vicoprofen ® 100 mg9.5 9 Vicoprofen ®  12 mg 9.5 9 Vicoprofen ®  40 mg/25 mg 9.5 9Vicoprofen ®  20 mg 9.5 9 Vicoprofen ®  10 mg 9.5 9 Vicoprofen ®  5 mg9.5 9 Vicoprofen ®   7.5 mg/200 mg 9.5 9 Tekturna ® 150 mg 9.5 9Saphris ®  5 mg 9.5 9 Potaba ® 500 mg 9.5 9 Myfortic ® 180 mg 9.5 9Maxalt-MLT ®  5 mg 9.5 9 EDARBYCLOR ®   40 mg/12.5 mg 9.5 9 Stribld ®550 mg 9.5 9 Stribld ® 1000 mg  9.5 9 Stribld ®   5 mg/1000 mg 9.5 9Stribld ®   5 mg/500 mg 9.5 9 Stribld ®  100 mg/1000 mg 9.5 9 Stribld ®  50 mg/1000 mg 9.5 9 Stribld ® 500 mg 9.5 9 Stribld ® 420 mg 9.5 9Stribld ®  1.0 mg 9.5 9 Stribld ® 150 mg/150 mg/ 9.5 9 200 mg/300 mgNorvir ® 100 mg 9.5 9 Kombiglyze XR ®   2.5 mg/1000 mg 9.5 9 JanumetXR ®  50 mg/500 mg 9.5 9 Atripla ® 600 mg/200 mg/ 9.5 9 300 mg Xifaxan ® 80 mg 8.7 8.5 Xifaxan ®  5 mg 8.7 8.5 Xifaxan ®  80 mg 8.7 8.5Xifaxan ®  60 mg 8.7 8.5 Xifaxan ® 100 mg 8.7 8.5 Xifaxan ®  4 mg 8.78.5 Xifaxan ®  10 mg 8.7 8.5 Xifaxan ®  80 mg 8.7 8.5 Xifaxan ®  30 mg8.7 8.5 Xifaxan ® 200 mg 8.7 8.5 Juvisync ® 100 mg/10 mg 8.7 8.5COARTEM ®  20 mg/120 mg 8.7 8.5 Vicodin ES ® 300 mg 8.7 8.5 Vicodin ES ®600 mg/25 mg 8.7 8.5 Vicodin ES ® 600 mg 8.7 8.5 Vicodin ES ® 1.25 mg 8.7 8.5 Vicodin ES ® 250 mg 8.7 8.5 Vicodin ES ® 200 mg 8.7 8.5 VicodinES ® 360 mg 8.7 8.5 Vicodin ES ® 500 mg 8.7 8.5 Vicodin ES ®   7.5mg/750 mg 8.7 8.5 Teveten HCT ®   600 mg/12.5 mg 8.7 8.5 Teveten ® 400mg 8.7 8.5 Norvir ® 100 mg 8.7 8.5 Lialda ® 1.2 g  8.7 8.5 DEPAKOTE ER ®250 mg 8.7 8.5 Baraclude ®  0.5 mg 8.7 8.5 Biaxin XL Filmtab ®  300 mg38 7.5 Biaxin XL Filmtab ® 500 mg 8 7.5 Viread ®  70 mg 7.9 7.5 Viread ® 8 mg 7.9 7.5 Viread ®  8 mg 7.9 7.5 Viread ®  10 mg 7.9 7.5 Viread ®  7mg 7.9 7.5 Viread ®  6 mg 7.9 7.5 Viread ®  5 mg 7.9 7.5 Viread ®  4 mg7.9 7.5 Viread ®  3 mg 7.9 7.5 Viread ®  2.5 mg 7.9 7.5 Viread ®  2 mg7.9 7.5 Viread ® 150 mg 7.9 7.5 Uloric ®  40 mg 7.9 7.5 Mevacor ®  20 mg7.9 7.5 Gleevec ® 100 mg 7.9 7.5 COUMAOIN ®  1 mg 7.9 7.5 Truvada ®  80mg 7.9 7.5 Truvada ® 145 mg 7.9 7.5 Truvada ® 300 mg/10 mg 7.9 7.5Truvada ® 200 mg 7.9 7.5 Truvada ®   4 mg/240 mg 7.9 7.5 Truvada ®   2mg/240 mg 7.9 7.5 Truvada ®   2 mg/240 mg 7.9 7.5 Truvada ®   1 mg/240mg 7.9 7.5 Truvada ® 1000 mg/20 mg  7.9 7.5 Truvada ® 750 mg/20 mg 7.97.5 Truvada ® 500 mg 7.9 7.5 Truvada ® 750 mg 7.9 7.5 Truvada ® 120 mg7.9 7.5 Truvada ® 400 mg 7.9 7.5 Truvada ® 100 mg 7.9 7.5 Truvada ® 10mg/320 mg/ 7.9 7.5 25 mg Truvada ® 320 mg/25 mg 7.9 7.5 Truvada ® 400 mg7.9 7.5 Truvada ® 240 mg 7.9 7.5 Truvada ®  200 mg/300 mg 7.9 7.5Tarka ®   2 mg/180 mg 7.9 7.5 Singulair ®  4 mg 7.9 7.5 Ranexa ® 500 mg7.9 7.5 Noroxin ® 400 mg 7.9 7.5 Janumet ®  50 mg/500 mg 7.9 7.5Tradjenta ® 0.75 mg  7.1 6.5 Tradjenta ®  20 mg 7.1 6.5 Tradjenta ®  30mg 7.1 6.5 Tradjenta ®  5 mg 7.1 6.5 Tradjenta ®  75 mg 7.1 6.5Tradjenta ®  4 mg 7.1 6.5 Tradjenta ®  40 mg 7.1 6.5 Tradjenta ®  50 mg7.1 6.5 Tradjenta ®  3 mg 7.1 6.5 Tradjenta ®  1 mg 7.1 6.5 Tradjenta ® 15 mg 7.1 6.5 Tradjenta ®  20 mg 7.1 6.5 Tradjenta ®  5 mg 7.1 6.5Samsca ®  15 mg 7.1 6.5 Propecia ®  1 mg 7.1 6.5 Prinzide ®   20 mg/12.5mg 7.1 6.5 Onglyza ®  2.5 mg 7.1 6.5 Isentress ®  25 mg 7.1 6.5 EDARBI ® 40 mg 7.1 6.5 Vicodin HP ®  40 mg 7.1 6.5 Vicodin HP ® 10/80 mg   7.16.5 Vicodin HP ® 250 mg 7.1 6.5 Vicodin HP ® 135 mg 7.1 6.5 Vicodin HP ®250 mg 7.1 6.5 Vicodin HP ® 300 mg/5 mg  7.1 6.5 Vicodin HP ® 100 mg 7.16.5 Vicodin HP ®  25 mg 7.1 6.5 Vicodin HP ®  10 mg 7.1 6.5 Vicodin HP ®100 mg 7.1 6.5 Vicodin HP ® 0.625 mg/5 mg   7.1 6.5 Vicodin HP ® 0.625mg/2.5 mg  7.1 6.5 Vicodin HP ® 0.45 mg/1.5 mg 7.1 6.5 Vicodin HP ® 100mg 7.1 6.5 Vicodin HP ® 360 mg 7.1 6.5 Vicodin HP ® 400 mg 7.1 6.5Vicodin HP ®  40 mg 7.1 6.5 Vicodin HP ®  35 mg 7.1 6.5 Vicodin HP ®  30mg 7.1 6.5 Vicodin HP ®  25 mg 7.1 6.5 Vicodin HP ®  10 mg/320 mg 7.16.5 Vicodin HP ®   5 mg/320 mg 7.1 6.5 Vicodin HP ®   320 mg/12.5 mg 7.16.5 Vicodin HP ® 320 mg 7.1 6.5 Vicodin HP ® 500 mg 7.1 6.5 Vicodin HP ®200 mg 7.1 6.5 Vicodin HP ® 180 mg 7.1 6.5 Vicodin HP ® 150 mg/10 mg/7.1 6.5 12.5 mg Vicodin HP ® 300 mg/10 mg/ 7.1 6.5 12.5 mg Vicodin HP ®300 mg/5 mg/ 7.1 6.5 25 mg Vicodin HP ® 300 mg/5 mg/ 7.1 6.5 12.5 mgVicodin HP ® 750 mg/20 mg 7.1 6.5 Vicodin HP ®  15 mg 7.1 6.5 VicodinHP ®   10 mg/660 mg 7.1 6.5 Vicodin ®    5 mg/500 mg 7.1 6.5 Simcor ®500 mg/20 mg 7.1 6.5 PRISTIQ ®  50 mg 7.1 6.5 PREMPRO ®  0.3 mg/1.5 mg7.1 6.5 Niaspan ® 500 mg 7.1 6.5 COMPLERA ® 200 mg/25 mg/ 7.1 6.5 300 mgAvelox ® 400 mg 7.1 6.5 Apriso ® 0.375 g   7.1 6.5 Adderall XR ®  5 mg7.1 6.5 Letairis ® 600 mg 6.7 6.5 Letairis ®  5 mg 6.35 6 Synthroid ® 300 mcg 6.3 6 Synthroid ®  200 mcg 6.3 6 Synthroid ®  175 mcg 6.3 6Synthroid ®  150 mcg 6.3 6 Synthroid ®  137 mcg 6.3 6 Synthroid ®  125mcg 6.3 6 Synthroid ®  112 mcg 6.3 6 Synthroid ®  100 mcg 6.3 6Synthroid ®  88 mcg 6.3 6 Synthroid ®  75 mcg 6.3 6 Synthroid ®  50 mcg6.3 6 Synthroid ®  40 mg 6.3 6 Synthroid ®  30 mg 6.3 6 Synthroid ®  20mg 6.3 6 Synthroid ®  15 mg 6.3 6 Synthroid ®  40 mg 6.3 6 Synthroid ® 2 mg 6.3 6 Synthroid ®  2 mg 6.3 6 Synthroid ®  6 mg 6.3 6 Synthroid ® 25 mcg 6.3 6 Proscar ®  5 mg 6.3 6 Oxycontin ®  10 mg 6.3 6 Intuniv ® 1 mg 6.3 6 Abilify discmelt ®  10 mg 6.3 6 Tasigna ®  80 mg 6.3 6Tasigna ® 200 mg 6.3 6 Tasigna ® 150 mg 6.3 6 Tasigna ® 140 mg 6.3 6Tasigna ® 300 mg/25 mg 6.3 6 Tasigna ®   300 mg/12.5 mg 6.3 6 Tasigna ®300 mg 6.3 6 Tasigna ® 200 mg 6.3 6 Tasigna ® 300 mg 6.3 6 Tasigna ® 200mg 6.3 6 Tasigna ®  0.9 mg 6.3 6 Tasigna ® 0.625 mg   6.3 6 Tasigna ®0.45 mg  6.3 6 Tasigna ® 150 mg 6.3 6 Tasigna ® 500 mg 6.3 6 Tasigna ®150 mg 6.3 6 Tasigna ® 100 mg 6.3 6 Tasigna ® 1000 mg  6.3 6 Tasigna ® 80 mg 6.3 6 Tasigna ® 100 mg/40 mg 6.3 6 Tasigna ®   70 mg/5600 IU 6.36 Tasigna ®  20 mg 6.3 6 Tasigna ® 125 mg 6.3 6 Tasigna ® 160 mg 6.3 6Tasigna ® 250 mg 6.3 6 Tasigna ®  60 mg 6.3 6 Tasigna ®  10 mg 6.3 6Tasigna ® 1000 mg/40 mg  6.3 6 Tasigna ® 1000 mg/20 mg  6.3 6 Tasigna ®150 mg 6.3 6 PREMARIN ®  0.3 mg 6.3 6 Nucynta ER ®  50 mg 6.3 6 Neoral ® 20 mg 6.3 6 Fosamax Plus D ®   70 mg/2800 IU 6.3 6 Fosamax ®  70 mg 6.36 DEPAKENE ® 250 mg 6.3 6 CLARINEX-D 12 HOUR ®   2.5 mg/120 mg 6.3 6Cardizem LA ® 120 mg 6.3 6 Biaxin Filmtab ® 250 mg 6.3 6 Advicor ® 500mg/20 mg 6.3 6 (500 mg extended- release niacin, and 20 mg of immediate-release lovastatin) Spiriva Handihaler ®  18 mcg 5.6 5 Xarelto ®  0.5 mg5.5 5 Xarelto ®  15 mg 5.5 5 Xarelto ®  4 mg 5.5 5 Xarelto ®  2 mg 5.5 5Xarelto ®  15 mg 5.5 5 Xarelto ®  10 mg 5.5 5 Sprygel ®  20 mg 5.5 5Nucynta ®  50 mg 5.5 5 Mavik ®  1 mg 5.5 5 Latuda ®  20 mg 5.5 5Januvia ®  25 mg 5.5 5 Femara ®  2.5 mg 5.5 5 CLARINEX ®  5 mg 5.5 5Azilect ®  0.5 mg 5.5 5 Trilipix ®  20 mg 5.5 5 Trilipix ®   4 mcg 5.5 5Trilipix ®   2 mcg 5.5 5 Trilipix ® 100 mg 5.5 5 Trilipix ®  20 mg 5.5 5Trilipix ® 150 mg/25 mg 5.5 5 Trilipix ® 150 mg/10 mg 5.5 5 Trilipix ®150 mg 5.5 5 Trilipix ® 100 mg/20 mg 5.5 5 Trilipix ®  4 mg 5.5 5Trilipix ®  2 mg 5.5 5 Trilipix ®  15 mg 5.5 5 Trilipix ®  10 mg 5.5 5Trilipix ® 10 mg/160 mg/ 5.5 5 25 mg Trilipix ® 5 mg/160 mg/ 5.5 5 25 mgTrilipix ® 10 mg/160 mg/ 5.5 5 12.5 mg Trilipix ®  80 mg 5.5 5Trilipix ® 100 mg 5.5 5 Trilipix ® 160 mg/25 mg 5.5 5 Trilipix ®   160mg/12.5 mg 5.5 5 Trilipix ®  60 mg 5.5 5 Trilipix ®  24 mcg 5.5 5Trilipix ®  45 mg 5.5 5 Tricor ®  48 mg 5.5 5 Tekamlo ® 150 mg/5 mg  5.55 Soriatane ®  10 mg 5.5 5 Reyataz ® 100 mg 5.5 5 Pentasa ® 250 mg 5.5 5Focalin XR ®  5 mg 5.5 5 EXFORGE HCT ® 5 mg/160 mg/ 5.5 5 12.5 mgDEXILANT ®  30 mg 5.5 5 DEPAKOTE ® 125 mg 5.5 5 Crixivan ® 100 mg 5.5 5Amturnide ® 150 mg/5 mg/ 5.5 5 12.5 mg Amitiza ®   8 mcg 5.5 5Zortress ®  4 mg 4.7 4.5 Zortress ® 0.25 mg  4.7 4.5 Livalo ®  1 mg 4.74.5 FANAPT ®  1 mg 4.7 4.5 DILAUDID ®  2 mg 4.7 4.5 Bystolic ®  2.5 mg4.7 4.5 Zocor ®  10 mg 4.7 4.5 Zocor ® 10/40 mg   4.7 4.5 Zocor ® 10/20mg   4.7 4.5 Zocor ® 100 mg 4.7 4.5 Zocor ®  50 mg 4.7 4.5 Zocor ®  290mcg 4.7 4.5 Zocor ®  9 mg 4.7 4.5 Zocor ®  6 mg 4.7 4.5 Zocor ®  3 mg4.7 4.5 Zocor ®  10 mg/160 mg 4.7 4.5 Zocor ®  80 mg 4.7 4.5 Zocor ®  30mg 4.7 4.5 Zocor ®  5 mg 4.7 4.5 Zemplar ®   1 mcg 4.7 4.5 Temodar ®  5mg 4.7 4.5 Tekturna HCT ®   150 mg/12.5 mg 4.7 4.5 Sustiva ®  50 mg 4.74.5 PRADAXA ®  75 mg 4.7 4.5 Linzess ®  145 mcg 4.7 4.5 Invega ®  1.5 mg4.7 4.5 EXFORGE ®   5 mg/160 mg 4.7 4.5 EMEND ®  40 mg 4.7 4.5 DIOVANHCT ®   80 mg/12.5 mg 4.7 4.5 CYMBALTA ®  20 mg 4.7 4.5 Afinitor ®  5 mg4.7 4.5 DALIRESP ® 500 mg 3.9 3.5 Zolinza ®  70 mg 3.9 3.5 Zolinza ®  60mg 3.9 3.5 Zolinza ®  50 mg 3.9 3.5 Zolinza ®  40 mg 3.9 3.5 Zolinza ® 30 mg 3.9 3.5 Zolinza ®  10 mg 3.9 3.5 Zolinza ®  10 mg 3.9 3.5Zolinza ® 100 mg 3.9 3.5 Vyvanse ®  20 mg 3.9 3.5 Vytorin ® 10/10 mg  3.9 3.5 Vimpat ®  50 mg 3.9 3.5 DYRENIUM ®  50 mg 3.9 3.5 DEPAKOTESPRINKLE ® 125 mg 3.9 3.5 CAPSULES COLCRYS ®  0.6 mg 3.9 3.5 Maxalt ® 10 mg 3.2 2.5 Maxalt ®  5 mg 3.2 2.5 Maxalt ®  5 mg 3.2 2.5 Abilify ® 2 mg 3.2 2.5 Zetia ®  10 mg 3.1 2.5 DIOVAN ®  40 mg 3.1 2.5 ¹Thesedimensions are based on a visual inspection of actual-size pictures ofthe medicines available in the PDR 2014 Edition of Nurse's DrugHandbook.

In one embodiment, the identification and sorting apparatus 901 (FIG.11A) is placed in front of each of the windows 1080 (except maybe thelast window opposite to the last ramp) to capture the medicines exitingor being ejected out of the sorter 1000. Based on the information inTable 3, for example, it is possible to know which medicines are mostprobable to exit which windows. This knowledge will facilitate orimprove the image analysis and computational time required foridentification of the medicines traveling through the identificationchamber 202 (FIG. 11A).

In some embodiments, additional image capturing devices 13 (in FIG. 1)are installed on the sorter 1000 (e.g., imagers P1-P6 in FIG. 13), forexample, on the top of the guide plate 1022 above ramps. A series oflights L1-L4 (only four shown in FIG. 13) may also be installed near theguide plate (above, below or in front of it) to facilitate the imagingof the medicines. Additional sensors may be installed near each of thewindows 1080 to count the medicines as they exit the sorter 1000. Theoutput of the imagers P1-P6 or the sensors would be fed to, imageprocessing device 21 (in FIG. 1) or the characterizer 520 (FIG. 7) ordisplayed on, a conventional digital display (e.g., 590 in FIG. 7). Toimage the medicines at the sorter 1000 stage, the entire or partialstructure of the sorter 1000 including the pad 1012 and the guide plate1022 (FIG. 13) may be made of transparent plastic or thermoplastic suchas Plexiglas® or Lexan®. In one embodiment, the transparent surface inthe sorter 1000 is constructed of scratch resistant, optical grade glasssuch as Corning® Gorilla® Glass.

Medicine Processing Machine

FIG. 18 shows an exemplary medicine-processing machine 1200 of thesubject technology in a side view illustrating its various modules. Themedicine processing machine includes medicine receiving module 1210which is designed to receive loose medicines or a mixture thereofdeposited in the medicine input receptacle 1214. From the module 1210,medicines are transported directly into the identification and sortingmodule 1220, which includes the identification and sorting system of thesubject technology. Medicines are sorted and identified in the module1220, from which they are distributed to the collections bins of themedicine collection/storage station 1230. A controller 1240 is coupledto each module within the medicine-processing machine 1200 and controlsthe interaction between each module. For example, the controller 1240reviews the output sorting and identification data from modules 1220 and1230 and displays them on the display module 1250 and cause the displaymodule to dispense discount coupons or receipts to the customer.

FIG. 19 illustrates a medicine distribution device 1250 which may beattached to each of the plurality of outlets 933 a-e of the lineardistribution manifold illustrated in FIG. 11B. The medicine distributiondevice 1250 includes a primary diverter 1251 and two secondary diverters1252 a and b. The medicine distribution device 1250 may includetertiary, quaternary, and so on, diverters to directing each medicine ofa specific brand or identity to a specific collection bin. Additionalimaging devices or sensors may be installed along the chutes of themedicine distribution device 1250 to track a medicine as it movestowards its correct collection bin. In one embodiment (FIG. 20), themedicine processing machine 1200 includes distribution belts orconveyors 1274 (FIG. 20) instead of the distribution device 1250 (FIG.19) for achieving essentially the same result, delivering an identifiedmedicine to a designated collection area 1278 (FIG. 20). The medicinedistribution device, belt or conveyor may form a complex network ofpaths and diverters facilitating the distribution of hundreds orthousands of medicines to different collection bins where each bin maycontain only a single type of medicine or medicines with the same activeingredients.

In an exemplary embodiment, the medicine distribution device 1250 (FIG.19) has four outlets leading to collections bins 904 for unknown orunidentified medicines, prescription (RX) medicines, over the counter(OTC) medicines, and crushed or powdered medicines. The mountingmechanisms 1253 for attaching the medicine collection bins to themedicine distribution device can be of a variety of devices including apivotal clamp, a sliding clamp, or a quick release fastener amongothers. The purpose of these mounting mechanisms 1253 is to physicallyattach the medicine collection bins 904 to the medicine distributiondevice 1250 while they are being filled with medicines. The controller1240 (FIG. 18) can monitor the number medicines going into any givencollection bin and stop the flow additional medicines to that bin if itis full. The controller can further display the bin being full on thedisplay module 1250, actuate a mechanism for the filled collection binto be automatically replaced by an empty bin, or automatically contact aservice operator to come and replace the filled bin.

In some embodiments, if a collection bin is designated to receive, forexample, medicines containing narcotics, the collection bin may includea deactivating agent (e.g., bleach) or an abuse-preventing agent toprevent the possibility of the medicines being stolen or abused fromthis bin. In another embodiment, the information about which bincontains what sorted medicines is kept secret or encrypted forpreventing possible abuse. In this case, the collection bins have nomarkings other than bar codes that will be read at a collection facilityby authorized personnel. This security process can be further enhancedby utilization of the media reader on the medicine-processing machine1200. In one embodiment, only an authorized personnel who would firstinsert a card to the media reader slot 1252 (FIG. 20) on the machine,which identifies him or her as a particular authorized person, canobtain access to the inside of the machine.

In one embodiment, a medicine-processing machine of the subjecttechnology acts as a drop box or a medicine deposition kiosk forreceiving unused, expired or recalled medications, in particular, unusedsolid and semisolid oral dosage forms. It may further be configured toaccept syrups and ampules or any other un-used, expired, recalled orpartially used medications safely and securely. In an embodiment wherethe medicine-processing machine of the subject technology acts as a dropbox, it is configured to separate medicines into Rx (prescriptionmedication), OTC (over the counter medication) or unknown/unidentified.The Rx and OTC medicines will then be transported to a medicationrecycling facility where the medicines are separate based on, forexample, their active ingredients. The unknown/unidentified medicineswill be identified in a lab, and their information will be added to thereference medicine image database/storage 31/510 (FIG. 1 or 7,respectively), which will facilitate the identification of thesemedicines at a later date. In another embodiment, the medicineprocessing machine has means for controlling the temperature of thecollection bins/storage area of the machine. In another embodiment, themedicine processing machine of the subject technology has means forkeeping any residual medication particles or dust inside the medicineprocessing machine. In this case, the machine includes a vacuum orsuctioning means for pulling the air from outside in and passing itthrough a hepa filter before sending the air out. This air suctioningand filtration action ensures that no residual particles or dust frommedicines are discharged from the medicine processing machine. Inanother embodiment, the medicine processing machine, in return fordepositing unused medicines, generates coupons or receipts that can beredeemed for cash or for receiving discounts at a store. In anotherembodiment, the medicine processing machine or kiosk allows the user toaccess their medicine deposit history on the machine's display unit oronline via a computer or a smart mobile phone.

In another embodiment, the medicine processing machine of the subjecttechnology includes a plurality of sorters 1000, medicine identificationchambers 21 and distribution mechanisms which are installed in parallelor in series (e.g., in cascade), which will allow for the identificationand sorting of thousands of medicines.

FIG. 20 illustrates an exemplary medicine sorting and storage system ordevice in which loose, mixed medicines 1262 are introduced to the systemby placing them on the surface 1262 to which shaker/vibrator 1265 isattached for facilitating the movement and feeding of the medicines intothe identification and sorting device. At the bottom of surface 1260 aconveyor belt 1264 picks the medicines and carries them upward fordelivery into a medicine identification chamber 1270. The conveyor belt1264 in the areas where medicines are carried has edges 1267 on bothsides (to form a groove) to prevent the medicines from falling off theconveyor belt. The conveyor belt may have multiple medicine carryinggrooves ending in multiple medicine identification chambers 1270. In anembodiment, the width of these grooves is between 2-15 cm or slightlywider than the largest medicine to be carried in them. In addition tothe edges 1267, plate 1268 may be installed above the conveyor belt toprevent medicines 1262 from pouring on and interfering with themedicines that are moving on the belt 1264. The plate 1268 may have anedge 1266 that prevents the balk medicines to get on the conveyor belt.Rather, the edge 1266 causes medicines to move individually as much aspossible or in a single file along the path towards the medicineidentification chamber. In addition, both the conveyor belt 1264 and theplate 1268 have shakers or vibrators 1265 to cause the medicine toseparate from one another and move individually on the belt. The shakingor vibration urges the piggybacked medicines to disengage from oneanother and to move in a single file and behind each other as they movetowards the medicine identification chamber.

Once the medicines are passed through the medicine identificationchamber 1270 and are identified, they are transferred onto conveyor belt1272 to be distributed or sorted to their desired containers. Thecontroller 1261 (which functions similar to controller 1240 in FIG. 18)ensures that proper diverters 1275 open to route each medicine to itsdesignated bin or container 1278. Additional imaging devices or sensorsmay be installed along conveyor belt 1272 to track a medicine as itmoves towards its correct collection bin 1278. Each diverter is actuatedby motor 1273 and the belt has edges 1274 to prevent medicines fromfalling off the conveyor belt. In one embodiment, the medicines to beidentified and sorted may be bottles of liquid medicines (e.g., syrups).In this case, the conveyor belt 1264 is placed be horizontal and themedicine identification chamber is designed such that the conveyor beltpasses through it while carrying the bottles to be identified. Thedevice may further include a robotic bottle-opener and a robotic arm topour the liquid medicines in appropriate containers once they areidentified.

Example I Separating Hazardous Drugs from Mixture of Drugs

Large quantities of mixed medicines accumulate daily at nursing home, ahospital and doctors office or pharmaceutical companies. A lot of thesemedicines may not be necessarily hazardous but the possibility that theymay have been mixed with hazardous medicines is high. The method anddevice of the subject technology allows for identification andseparation of hazardous medicines from non-hazardous ones. Thisseparation facilitates the proper disposal of these hazardous medicinesand helps reducing the adverse impact of these medicines on theenvironment and human health.

Thus, in this example, the device of the subject technology isconfigured to separate hazardous medicines from non-hazardous ones.Exemplary hazardous medicines are listed in Table 4 below (which is areproduction of the NIOSH List of hazardous medicines announced by theU.S. National Institute for Occupational Safety and Health in 2012). Theinformation about the physical (including their shapes, colors, surfacelines, imprints, markings, debosses, embosses, grooves, writing or etc.)and chemical features of these medicines as well as their images arecollected and stored in the system (e.g., the image processing unit orthe reference image database) of the subject technology. Preferably, themedicine identification chamber or unit of the subject technology isused to generate multiple reference images for each of these medicines.In accordance to the method and device of the subject technology, atarget medicine is then processed for identification by comparing itsdigital image(s) with the digital images of the reference medicines.

Upon introduction of medicines to the device of the subject technology,hazardous medicines are quickly identified and separated from thenon-hazardous medicines. The hazardous medicines are then subjected toproper disposal while the non-hazardous medicines can be shipped tolandfills.

TABLE 4 Sample List of Drugs that Should be Handled as Hazardous DrugSource AHFS Pharmacologic-therapeutic classification Acitretin 7 88:04Vitamin A Aldesleukin 4, 5 10:00 Antineoplastic agents Ambrisentan 724:12.92 Vasodilating agents, miscellaneous Alefacept 6 84:92 Skin andmucous membrane agents, miscellaneous Alitretinoin 3, 4, 5 84:92 Skinand mucous membrane agents, miscellaneous Altretamine 1, 2, 3, 4, 510:00 Antineoplastic agents Amsacrine 3, 5 Not in AHFS (antineoplasticagent) Anastrozole 1, 5 10:00 Antineoplastic agents Arsenic trioxide 1,2, 3, 4, 5 10:00 Anti neoplastic agents Asparaginase 1, 2, 3, 4, 5 10:00Antineoplastic agents Azacitidine 3, 5 10:00 Anti neoplastic agentsAzathioprine 2, 3, 5 92:44 Immunosuppressant agents BacillusCalmette-Guerin 1, 2, 4 80:12 Vaccines (BCG) Bendamustine HCl 7 10:00Antineoplastic agents Bexarotene 2, 3, 4, 5 10:00 Anti neoplastic agentsBicalutamide 1, 5 10:00 Anti neoplastic agents Bleomycin 1, 2, 3, 4, 510:00 Antineoplastic agents Bortezomib 6 10:00 Antineoplastic agentsBosentan 6 24:12.92 Vasodilating agents, miscellaneous Busulfan 1, 2, 3,4, 5 10:00 Antineoplastic agents Cabergoline 7 28:36.20.04Ergot-derivative dopamine receptor agonists Capecitabine 1, 2, 3, 4, 510:00 Antineoplastic agents Carbamazepine 7 28:12.92 Anticonvulsants,miscellaneous Carboplatin 1, 2, 3, 4, 5 10:00 Antineoplastic agentsCarmustine 1, 2, 3, 4, 5 10:00 Antineoplastic agents Cetrorelix acetate5 92:40 Gonadotropin-releasing hormone antagonists Chlorambucil 1, 2, 3,4, 5 10:00 Antineoplastic agents Chloramphenicol 1, 5 8:12.08Chloramphenicols Choriogonadotropin alfa 5 68:18 Gonadotropins Cidofovir3, 5 8:18.32 Nucleosides and nucleotides Cisplatin 1, 2, 3, 4, 5 10:00Antineoplastic agents Cladribine 1, 2, 3, 4, 5 10:00 Antineoplasticagents Clofarabine 6 10:00 Antineoplastic agents Clonazepam 7 28:12.08Benzodiazepines Colchicine 5 92:16 Antigout agents Cyclophosphamide 1,2, 3, 4, 5 10:00 Antineoplastic agents Cyclosporin 1 92:44Immunosuppressive agents Cytarabine 1, 2, 3, 4, 5 10:00 Antineoplasticagents Dacarbazine 1, 2, 3, 4, 5 10:00 Anti neoplastic agentsDactinomycin 1, 2, 3, 4, 5 10:00 Anti neoplastic agents Dasatinib 610:00 Antineoplastic agents Daunorubicin HCl 1, 2, 3, 4, 5 10:00Antineoplastic agents Decitibine 6 10:00 Antineoplastic agents Degarelix7 10:00 Anti neoplastic agents Denileukin 3, 4, 5 10:00 Antineoplasticagents Diethylstilbestrol 5 Not in AHFS (nonsteroidal syntheticestrogen) Dinoprostone 5 76:00 Oxytocics Docetaxel 1, 2, 3, 4, 5 10:00Antineoplastic agents Doxorubicin 1, 2, 3, 4, 5 10:00 Antineoplasticagents Dronedarone HCl 7 24:04.04 Antiarrythmics Dutasteride 5 92:085-alpha reductase inhibitors Entecavir 6 8:18.32 Nucleosides andnucleotides Epirubicin 1, 2, 3, 4, 5 10:00 Antineoplastic agentsErgonovine/ 5 76:00 Oxytocics methylergonovine Estradiol 1, 5 68:16.04Estrogens Estramustine phosphate 1, 2, 3, 4, 5 10:00 Antineoplasticagents Estrogen-progestin 5 68:12 Contraceptives combinations Estrogens,conjugated 5 68:16.04 Estrogens Estrogens, esterifled 5 68:16.04Estrogens Estrone 5 68:16.04 Estrogens Estropipate 5 68:16.04 EstrogensEtoposide 1, 2, 3, 4, 5 10:00 Antineoplastic agents Everolimus 7 10:00Anti neoplastic agents Exemestane 1, 5 10:00 Antineoplastic agentsFinasteride 1, 3, 5 92:08 5-alpha reductase inhibitors Floxuridine 1, 2,3, 4, 5 10:00 Antineoplastic agents Fludarabine 1, 2, 3, 4, 5 10:00Antineoplastic agents Fluorouracil 1, 2, 3, 4, 5 10:00 Anti neoplasticagents Fluoxymesterone 5 68:08 Androgens Flutamide 1, 2, 5 10:00Antineoplastic agents Fulvestrant 5 10:00 Antineoplastic agentsGanciclovir 1, 2, 3, 4, 5 8:18.32 Nucleosides and nucleotides Ganirelixacetate 5 92:40 Gonadotropin-releasing hormone antagonists Gemcitabine1, 2, 3, 4, 5 10:00 Antineoplastic agents Gemtuzumab ozogamicin 1, 3, 4,5 10:00 Antineoplastic agents Gonadotropin, chorionic 5 68:18Gonadotropins Goserelin 1, 2, 5 10:00 Antineoplastic agents Hydroxyurea1, 2, 3, 4, 5 10:00 Antineoplastic agents Idarubicin 1, 2, 3, 4, 5 10:00Antineoplastic agents Ifosfamide 1, 2, 3, 4, 5 10:00 Antineoplasticagents Imatinib mesylate 1, 3, 4, 5 10:00 Antineoplastic agentsIrinotecan HCl 1, 2, 3, 4, 5 10:00 Antineoplastic agents Ixabepilone 710:00 Anti neoplastic agents Leflunomide 3, 5 92:36 Disease modifyingantirheumatic agents Lenalidomide 6 92:20 Biologic response modifiersLetrozole 15 10:00 Antineoplastic agents Leuprolide acetate 1, 2, 510:00 Antineoplastic agents Lomustine 1, 2, 3, 4, 5 10:00 Antineoplasticagents Mechlorethamine 1, 2, 3, 4, 5 10:00 Antineoplastic agentsMedroxyprogesterone 6 68:32 Progestins acetate Megestrol 1, 5 10:00Antineoplastic agents Melphalan 1, 2, 3, 4, 5 10:00 Antineoplasticagents Menotropins 5 68:18 Gonadotropins Mercaptopurine 1, 2, 3, 4, 510:00 Antineoplastic agents Methotrexate 1, 2, 3, 4, 5 10:00 Antineoplastic agents Methyltestosterone 5 68:08 Androgens Mifepristone 576:00 Oxytocics Mitomycin 1, 2, 3, 4, 5 10:00 Antineoplastic agentsMitotane 1, 4, 5 10:00 Antineoplastic agents Mitoxantrone HCl 1, 2, 3,4, 5 10:00 Antineoplastic agents Mycophenolate mofetil 1, 3, 5 g2:44Immunosuppressive agents Mycophenolic acid 7 92:44 Immunosuppressiveagents Nafarelin 5 68:18 Gonadotropins Nelarabine 6 10:00 Antineoplasticagents Nilotinib 7 10:00 Anti neoplastic agents Ni lutamide 1, 5 10:00Antineoplastic agents Oxaliplatin 1, 3, 4, 5 10:00 Antineoplastic agentsOxcarbazepine 7 28:12.92 Anticonvulsants, miscellaneous Oxytocin 5 76:00Oxytocics Paclitaxel 1, 2, 3, 4, 5 10:00 Antineoplastic agentsPalifermin 6 84:16 Cell stimulants and proliferants Paroxetine** 6, 728:16.04.20 Selective serotonin uptake inhibitors Pazopanib HCl 7 10:00Antineoplastic agents Pegaspargase 1, 2, 3, 4, 5 10:00 Anti neoplasticagents Pemetrexed 6 10:00 Antineoplastic agents Pentamidine isethionate1, 2, 3, 5 8:30.92 Antiprotozoals, miscellaneous Pentetate calcium 6 Notin AHFS trisodium^(††) Pentostatin 1, 2, 3, 4, 5 10:00 Antineoplasticagents Phenoxybenzamine HCl 7 12:16.04.04 Non-selective alpha-adrenergicblocking agents Pipobroman 3, 5 Not in AHFS (antineoplastic agent)Plerixafor 7 20:16 Hematopoietic agents Podofilox 5 84:92 Miscellaneousskin and mucous membrane agents (mitotic inhibitor) Podophyllum resin 584:92 Skin and mucous membrane agents, miscellaneous Pralatrexate 710:00 Antineoplastic agents Procarbazine 1, 2, 3, 4, 5 10:00Antineoplastic agents Progesterone 5 68:32 Progestins Progestins 5 68:12Contraceptives Raloxifene 5 68:16.12 Estrogen agonists-antagonistsRasagiline mesylate 6 28:36 Antiparkinsonian agents Ribavirin 1, 2, 58:18.32 Nucleosides and nucleotides Risperidone 6 28:16.08.04 Atypicalantipsychotics Romidepsin 7 10:00 Antineoplastic agents Sirolimus 692:44 Immunosuppressive agents Sorafenib 6 10:00 Antineoplastic agentsStreptozocin 1, 2, 3, 4, 5 10:00 Antineoplastic agents Sunitinib malate6 10:00 Antineoplastic agents Tacrolimus 1, 5 92:44 Immunosuppressiveagents Tamoxifen 1, 2, 5 10:00 Antineoplastic agents Televancin 78:12.28.16 Glycopeptides Temozolomide 3, 4, 5 10:00 Antineoplasticagents Temsirolimus 7 10:00 Antineoplastic agents Teniposide 1, 2, 3, 4,5 10:00 Antineoplastic agents Testolactone 5 10:00 Antineoplastic agentsTestosterone 5 68:08 Androgens Tetracycline HCl 7 8:12.24 TetracyclinesThalidomide 1, 3, 5 92:20 Biologic response modifiers Thioguanine 1, 2,3, 4, 5 10:00 Antineoplastic agents Thiotepa 1, 2, 3, 4, 5 10:00Antineoplastic agents Topotecan 1, 2, 3, 4, 5 10:00 Antineoplasticagents Toremifene citrate 1, 5 10:00 Antineoplastic agents Tretinoin 1,2, 3, 5 84:16 Cell stimulants and proliferants Trifluridine 1, 2, 552:04.20 Antivirals Triptorelin 5 10:00 Anti neoplastic agents Uracilmustard 3, 5 Not in AHFS (antineoplastic agent) Valganciclovir 1, 3, 58:18.32 Nucleosides and nucleotides Valproic acid/ 7 28:12.92Anticonvulsants, miscellaneous divalproex Na Valrubicin 1, 2, 3, 5 10:00Antineoplastic agents Vidarabine 1, 2, 5 Not in AHFS Vigabatrin 728:12.92 Anticonvulsants, miscellaneous Vinblastine sulfate 1, 2, 3, 4,5 10:00 Antineoplastic agents Vincristine sulfate 1, 2, 3, 4, 5 10:00Antineoplastic agents Vinorelbine tartrate 1, 2, 3, 4, 5 10:00Antineoplastic agents Vorinostat 6 10:00 Antineoplastic agentsZidovudine 1, 2, 5 8:18:08 Antiretroviral agents Ziprasidone HCl 728:16.08.04 Atypical antipsychotics Zoledronic acid 7 92:24 Boneresorption inhibitors Zonisamide 6 28:12.92 Anticonvulsants,miscellaneous 1. The NIH Clinical Center, Bethesda, MD (Revised August2002). The NIH Health Clinical Center Hazardous Drug (HD) list is partof the NIH Clinical Center's hazard communication program. It wasdeveloped in compliance with the OSHA hazard communication standard [29CFR 1910.1200] as it applies to hazardous drugs used in the workplace.The list is continually revised and represents the diversity of medicalpractice at the NIH Clinical Center; however, its content does notreflect an exhaustive review of all FDA-approved medications that may beconsidered hazardous, and it is not intended for use outside the NJH. 2.The Johns Hopkins Hospital, Baltimore, MD (Revised September 2002). 3.The Northside Hospital, Atlanla, GA (Revised August 2002). 4. TheUniversity of Michigan Hospitals and Health Centers. Ann Arbor, MI(Revised February 2003) 5. This sample listing of hazardous drugs wascompiled by the Pharmaceutical Research and Manufacturers of America(PhRMA) using information from the AHFS DI monographs published by ASHPin selected AHFS Pharmacologic-Therapeutic Classification categories[ASHP/AHFS DI 2003] and applying the definition for hazardous drugs. Thelist also includes drugs from other sources that satisfy the definitionfor hazardous drugs [PDR 2004; Sweetrnan 2002; Shepard 2001; Schardein2000; REPROTOX 2003]. Newly approved drugs that have structures ortoxicological profiles that mimic the drugs on this list should also beincluded. This list was revised in June 2004. 6. NIOSH addition 2010updated using ASHP/AHFS DI 2010. 7. NIOSH addition 2012 updated usingASHP/AHFS DI 2011. **2010, Paroxetine HCl; 2012, Paroxetine mesylale^(††)Refers to non-radio-labeled formulation only.Radio-Pharmaceuticals that are Regulated by Nuclear RegulatoryCommission

Drug AHFS Pharmacologic-therapeutic classification ibritumomab tiuxetan10:00 Antineoplastic agents tositumomab 10:00 Antineoplastic agentsDrugs that are Currently not Available in the United States‡

Drug AHFS Pharmacologic-therapeutic classification dienestrol 68:16.04Estrogens interferon alfa n 1 10:00 Antineoplastic agents perphosphamideNot in AHFS (antineoplastic agent) piritrexim isethionate Not in AHFS(anti neoplastic agent) plicamycin Not in AHFS (anti neoplastic agent)prednumustine Not in AHFS (anti neoplastic agent) raltitrexed Not inAHFS (anti neoplastic agent) trimetrexate glucuronate 8:30.92Miscellaneous antiprotozoals vindesine Not in AHFS (anti neoplasticagent) ‡The NIOSH hazardous drug list is based on approvals by the U.S.FDA. These drugs are not approved by the U.S. FDA and are no longeravailable in the U.S. However, some may be available in other countries.

It is submitted that the subject technology has been shown and describedin what is considered to be exemplary embodiments. It is recognized,however, that departures may be made within the scope of the subjecttechnology and that obvious modifications will occur to a person skilledin the art. With respect to the above description then, it is to berealized that the optimum dimensional relationships for the parts of thesubject technology, to include variations in size, materials, shape,form, function and manner of operation, assembly and use, are deemedreadily apparent and obvious to one skilled in the art, and allequivalent relationships to those illustrated in the drawings anddescribed in the specification are intended to be encompassed by thepresent subject technology.

Therefore, the foregoing is considered as illustrative only of theprinciples of the subject technology. Further, since numerousmodifications and changes will readily occur to those skilled in theart, it is not desired to limit the subject technology to the exactconstruction and operation shown and described, and accordingly, allsuitable modifications and equivalents may be resorted to, fallingwithin the scope of the subject technology.

What is claimed is:
 1. A computer implemented method for sorting a plurality of different medicines, the method comprising the steps of (i) queuing the different medicines, wherein the queuing step comprises receiving and sorting the plurality of different medicines for identification, (ii) identifying the different medicines, after said queuing wherein the identifying step comprises an image capturing and an image processing for identifying each of the different medicines, (iii) sorting the different medicines, after said identifying wherein the sorting step comprises routing the identified medicines to separate locations.
 2. The method of claim 1, wherein the image processing comprises a reference image database comprising reference images of the different medicines or reference images of at least a portion of each of the reference medicines or a combination thereof, wherein said portion comprises edges and lines or a physical feature comprising shape, color, surface line, imprint, marking, deboss, emboss, groove or writing.
 3. The method of claim 2, wherein the image processing further comprises an algorithm for detecting, matching and/or classifying each of the different medicines based on the reference images.
 4. The method of claim 3, wherein the algorithm comprises squared difference, normalized squared difference, cross correlation, normalized cross correlation, correlation coefficient, normalized correlation coefficient, neural network, Bayesian belief network, support vector machines, fuzzy logic, Hidden Markov model.
 5. A device for sorting a plurality of different medicines, said device comprising a sorting an queuing unit for receiving and sorting the plurality of the different medicines for identification; an identification unit, wherein the identification unit comprises an image capturing and an image processing, for identifying each of the different medicines after said receiving and sorting; and a sorting unit for routing the identified different medicines to separate locations after said identification.
 6. The device of claim 5, wherein the image processing comprises a reference image database comprising reference images of the different medicines or reference images of at least a portion of each of the reference medicines, wherein said portion comprises edges and lines or a physical feature comprising shape, color, surface line, imprint, marking, deboss, emboss, groove or writing.
 7. The device of claim 6, wherein the image processing further comprises and an algorithm for detecting, matching and/or classifying each of the different medicines based on the reference images.
 8. The device of claim 7, wherein the algorithm comprises squared difference, normalized squared difference, cross correlation, normalized cross correlation, correlation coefficient, normalized correlation coefficient, neural network, Bayesian belief network, support vector machines, fuzzy logic or Hidden Markov model.
 9. A computer implemented method for separating hazardous and/or government regulated medicines from non-hazardous medicines, the method comprising the steps of (i) queuing the medicines, wherein the queuing step comprises receiving and sorting the medicines for identification, (ii) identifying the hazardous and/or government regulated medicines, after said queuing wherein the identifying step comprises an image capturing and an image processing for identifying each of the hazardous and/or government regulated medicines, (iii) sorting the hazardous and/or government regulated medicines, after said identifying wherein the sorting step comprises routing the identified hazardous and/or government regulated medicines to a separate location from the non-hazardous medicines for disposal.
 10. The method of claim 9, wherein the image processing comprises a reference image database comprising reference images of the hazardous and/or government regulated medicines or reference images at least a portion of each of the hazardous and/or government regulated medicines or a combination thereof, wherein said portion comprises edges and lines or a physical feature comprising shape, color, surface line, imprint, marking, deboss, emboss, groove or writing.
 11. The method of claim 10, wherein the image processing further comprises an algorithm for detecting, matching and/or classifying each of the hazardous and/or government regulated medicines based on the reference images.
 12. The method of claim 11, wherein the algorithm comprises squared difference, normalized squared difference, cross correlation, normalized cross correlation, correlation coefficient, normalized correlation coefficient, neural network, Bayesian belief network, support vector machines, fuzzy logic or Hidden Markov model.
 13. A device for separating hazardous and/or government regulated medicines from non-hazardous medicines, said device comprising a sorting and queuing unit for receiving and sorting the medicines for identification; an identification unit, wherein the identification unit comprises an image capturing and an image processing, for identifying each of the hazardous and/or government regulated medicines after said receiving and sorting; and a sorting unit for routing the identified hazardous and/or government regulated medicines to separate locations from the non-hazardous medicines for disposal after said identification.
 14. The device of claim 13, wherein the image processing comprises a reference image database comprising reference images of the hazardous and/or government regulated medicines or reference images of at least a portion of each of the hazardous and/or government regulated medicines or a combination thereof, wherein said portion comprises edges and lines or a physical feature comprising shape, color, surface line, imprint, marking, deboss, emboss, groove or writing.
 15. The device of claim 14, wherein the image processing further comprises an algorithm for detecting, matching and/or classifying each of the hazardous and/or government regulated medicines based on the reference images.
 16. The device of claim 15, wherein the algorithm comprises squared difference, normalized squared difference, cross correlation, normalized cross correlation, correlation coefficient, normalized correlation coefficient, neural network, Bayesian belief network, support vector machines, fuzzy logic or Hidden Markov model. 